Trion held their latest Friday RIFT Livestream on 22nd January 2016, with information relating to engineering and tech. The stream goes over roles, bug & code testing processes, info on some of the big engineering projects over the past year as well as some 2016 projects in the works.
Click here for a replay of the 22nd January 2016 RIFT Livestream video on twitch.tv. RIFT portion of the video starts at 59min 33sec in.
Black/Normal Text = Livestream Summary.
[Dark Blue Text] = My own comments. i.e. May not be indicative of actual content.
Trion had their 22nd January 2016 Livestream with Producer Amber “Gingers” Alexander, Lead Engineer Dan “Snedhepl” Hollinger and Senior Community Relations Manager Eric “Ocho” Cleaver.
Table of Contents:
- Role Introductions. Producer Gingers and Lead Engineer Snedhepl’s roles.
- Engineering and Tech. Bugs, code-testing and the general engineering process.
- Big Engineering Projects 2015/16. Chat reporting expanded, GUD System, tech debt/refactoring and major tech announcements in the next couple of weeks!
- Livestream Giveaways. With added Hide & Seek Pinata Party!
As always, any additions or changes mentioned below can be subject to change prior to hitting Live.
Both Gingers and Snedhepl briefly talked about their roles in the game.
As Producer, Gingers handles most of the scheduling for RIFT and coordinates tasks between different teams to ensure that all the pieces for each project fit in a timely manner. Each team lead can also talk to Gingers to push things back to a more reasonable workload. Gingers also maintains her role as a Senior UI Artist on top of being the Producer.
Lead Engineer Snedhepl
As Lead Engineer, Snedhepl does a bit of everything. He actively codes every day just as the other engineers, but he is also responsible for coordinating schedules and consulting with designers with their requests to make sure that what the designers want is technically feasible or not.
Engineering and Tech
The bulk of the livestream talked about engineering with info on recently created systems and bug fixes.
- Devs design pitch to Game Director Archonix.
- Once a pitch is agreed upon, Snedhepl checks its technical viability. If more research is required, another meeting will be scheduled.
- Snedhepl will discuss options with various people involved in the design pitch if some part is not viable.
- A list of tasks is determined and sent to their project management software to work out a schedule.
Tidbit: RIFT engineers primarily use C++, although they occasionally have bits of code in C Sharp or Python. They all work from the same code base.
Trion is notified of bugs in a variety of ways, from QA (quality assurance), forums, in-game reports and even feedback from devs playing the game.
- Importance of Reproduction. Being able to reproduce a bug is extremely important when it comes to quickly being able to identify and fix it. The more info players provide in terms of consistent reproduction steps, the better!
- Priorities. Once an internal bug report is written up, it will be prioritized amongst other bugs and workload. Snedhepl will occasionally consult with Archonix, Gingers or related devs to see how important a bug is for better prioritization.
- Occasionally trivial bugs that stick around long enough will go up the chain like the color bug, which Trion has tried to solve many times.
Akylios’ Platform Bug
One specific bug mentioned on the Livestream was Akylios’ platforms, which would randomly disappear or have the graphic linger. The reason engineers took so long to track down the bug was because they just couldn’t reproduce the bug on their internal client. It wasn’t until a recent report regarding the Instant Adventure version of the fight was submitted that Trion was finally able to reproduce the bug.
It turned out that there was a distance limiter applied to the initial platform animation. There’s a flag that designers can set to ignore culling distance, and that was set so the platforms wouldn’t disappear.
One of the most annoying bugs to fix involved the Minions System when it was first released. Somehow it was possible for database messages to arrive out of order. This ended up wiping players’ minion-related progress by anywhere from a few minutes to several hours.
It took approximately 2 weeks to track down and fix.
Snedhepl explained the recent Masteries Bug in detail. First the tl;dr from Vladd in the chat channel: Basically the “contextual holder (which triggers different abilities based on conditions) doesn’t play nice with ability queuing.”
And now the long version:
Trion wanted to make the cooldown permanently display on the Level 65 Mastery (Soul Stream/etc) abilities. However, these Masteries are actually made up of 3 abilities: a base ability and then two underlying abilities that are triggered based on the situation (target friendly/yourself = heal; target enemy = damage).
The two triggered abilities had cooldowns that weren’t being displayed, so Trion decided to give them their rightful cooldowns. Tests on the internal client gave the green light to put the changes to Live.
Turns out when you are dealing with the base ability on Live and there’s a little more latency (compared to internal’s <5ms latency), something weird happens.
- If you are on the global cooldown (‘gcd’) and you use the base ability, the base ability ends up going on gcd. This is not supposed to happen, because the base ability is only there to trigger one of the underlying abilities.
- Once your normal abilities are off the gcd, the base ability is actually still on gcd. However, the ability icon in your ability bar shows your underlying ability’s cooldown (which is off-gcd), not the base ability (which is still on gcd).
- Thus when you tried to use your Mastery ability (Soul Stream/etc), you were unable to do so because the game is still waiting for the base ability to go off-gcd.
Trion ended up reverting the changes to remove the bug.
Testing is done locally with an internal client that has special permissions and commands not available on the Live client.
- Once something has been changed, Snedhepl will compile the code locally.
- It will then be run on his computer where he will jump on and perform various actions to test the code.
- He’ll then check it in so that the code is available for internal servers the next day.
- QA will then hammer away at the changes on the internal servers for some broader testing.
[You then have the Public Test Shard for an even broader field for certain additions to the game.]
Remote Administration Tool
The game employs a Remote Administration Tool (“RAT”) that allows the engineers to enable/disable certain features on the Live game without requiring a server restart (like disabling a zone event or instant adventure). More existing features are being added to RAT over time, and whenever Trion adds a new feature, they try to build it so that it can be used with RAT.
Big Engineering Projects in 2015/16
Snedhepl talked about some of the big projects that were completed in 2015 as well as brief mention of projects being worked on in 2016.
Chat Reporting Expanded (2015)
In 2015, Trion expanded on chat reporting with some additional features.
How Chat Reporting Works
- Whenever you report someone, that person gets a ‘black mark’ put on their account and are auto-ignored.
- After reaching a threshold of black marks, the CS team is sent an e-mail asking them to look into the player. The e-mail includes a recent chat log for the player.
- If the player’s black marks reach a second, higher-tier threshold, they will automatically be muted for a certain amount of time until CS takes a look at their case. Actions also include permanent muting or outright bans depending on a variety of factors.
- ‘Black marks’ will decay over time, although Trion keeps a record of how many times players have given you black marks.
- Trion also has a record of all players who report others just in case someone tries to manipulate the system to harass other players.
- You won’t be able to see your black marks, and black marks are only generated from multiple, different players reporting – not from a single player reporting the same person multiple times.
GUD System (2015)
The GUD System (‘Good’) was added in 2015 to simplify the process of adding region-wide features. Prior to the creation of the GUD System, making systems like Minions and Wardrobe collections that were region-wide was incredibly annoying and took a long time.
With the GUD System in place, it will now be much easier and faster for Trion to add features that are region-wide. The introduction of the GUD System is what enabled Trion to create the recently-added Affinity System for patrons.
Tech Improvements/Refactoring (2016)
Over time, the game incurs what is known as ‘tech debt‘, which is basically when engineers do something which – at the time – was in a less than ideal way, or which is discovered to be less than ideal later on. One way to reduce ‘tech debt’ is a process known as refactoring.
Refactoring is where you take an existing piece of code that was perhaps created by a previous engineer with parts you don’t understand (or with so much code bloat that it looks like a monstrosity), and you fix it. This may include rewriting some of the code or occasionally requires rebuilding the piece of code from scratch.
Refactoring and reducing tech debt has always been done in RIFT, but in 2016 the engineering team will begin to have specific schedules dedicated to this task. Previously it was done whenever the engineers stumbled upon something that needed refactoring whilst they were working on a project. This pulled their time and attention away from the actual project/task they were working on.
In short: Bugs will get solved faster, and new features will likely be built faster.
Other Major Tech Announcements (2016)
Trion mentioned briefly during the livestream that other major tech announcements would be announced in a couple of weeks.
No further information was provided about the “major tech announcements”, although a small exchange between CM Ocho and Snedhepl might hint at significant optimizations.
At some point CM Ocho was reading player questions about optimizations and the following exchange occured:
CM Ocho: And there’s a lot of questions about optimizations. Now, that would be a pretty big announcement wouldn’t it?
Snedhepl: That would be a pretty big announcement.
CM Ocho: There we go. Definitely something we’re interested in.
Unconfirmed whether there will be any big optimizations as part of the announcements being made, but you can look forward to them in a couple of weeks’ time.
Gingers and Snedhepl will appear in another livestream in the near future for further UI and Engineering-related info.
- Archonix Letter Delayed. Archonix’s RIFT Producer Letter will possibly be delayed until some time in early February.
- Snedhepl Letter. Lead Engineer Snedhepl will have his own Letter posted up around early February as well. [Unknown subject, although it might relate to the major tech announcements that are going to be announced in a couple of weeks.]
- PTS ETA. Vladd mentioned that the Public Test Shard would probably be back online next Monday.
- New Primalist Souls Being Worked On. In the chat channel during the stream, Vladd mentioned that he has been actively working this week on either balancing existing Primalist souls, or the four new Primalist souls. [It was unclear from the chat messages which he was referring to. From some recent datamining, it’s probably the new souls.] Unless it has changed, the new souls slated for Primalists should cover Ranged ST Burst DPS, Melee ST Burst DPS, Support, and AoE/Raid Healer. [r]
- No Female Beard. Unfortunately, Archonix mentioned that Trion does not have plans to give Female Dwarves their beards.
- Assault on Bronze Tomb Issues. CM Ocho is looking at the Assault on Bronze Tomb warfront and discussing it with designers. He is keen to look at a consolidated thread that only talks about issues specifically regarding the warfront.
- Further Player Questions:
- Search function to RIFT Store main page? On list and something they want to do, but will take some time as the client isn’t fully aware of everything on the store all at once.
- Trion will take a look at problems with using “@” symbol for EU players. (i.e. some players have to paste the symbol into chat instead of just using SHIFT+2).
- Can take a look at Sorting for Guild Bank. No promises.
- Livestream times. Trion wants to keep a constant schedule for their games to avoid confusion. However, when they add new games to the livestreams in the future, they will take a look at holding the stream whilst considering EU times.
Livestream Channel Giveaways
During the Livestream, a handful of livestream chat channel viewers received:
- 1x Spooky the Nightmare Budgie to 3 livestream viewers.
- 5N0ZZ83RR135, Sy5t3m_3rr0r and PhobusPT.
Congratulations to all the winners!
You should receive information about your prizes in the ‘Inbox’ or ‘Other’ Tab in your Twitch inbox within the next few days.
Hide & Seek Pinata Party
During the livestream, Gingers went to various zones on Faeblight, showing a small area of her surroundings in a hide & seek-style event. She threw Planetouched Pinatas which reward up to 10 random players within a 30m range, random lockbox-style loot.
She spawned 12 Party Pinatas in total, effectively spreading up to 120 instances of random lockbox loot from the Jolt Supply Crate.
(From Left to Right: Gingers, Snedhepl and CM Ocho.)
Quite a bit of interesting information in regards to the tech and engineering side of the game. The GUD System has been an excellent addition to the game, and the work on chat reporting is most welcome. The extra scheduling for Refactoring will likely speed up bug fixes in the future and adding more features to the Remote Administration Tool will hopefully reduce the number of off-schedule server restarts.
Trion stated that Gingers and Snedhepl will be appearing on another livestream in the near future for further UI and engineering-related info. Perhaps Trion will release further information about those “major tech announcements” they have slated this year during that livestream.
Past Livestream Summaries can be accessed here.