In our Developer Interview series of posts we are continuing to introduce our team to you. Today we want to share an interview with our Quality Assurance department. We've sat down with Kieren and Lionel (who have already run through the content of our new The Fall of Oriath expansion a dozen times) and have asked them questions about their jobs in QA.

Hi! Thanks for the interview! Please, introduce yourselves and tell us about your history in gaming.

Kieren: Hi, I'm Kieren. I've had a good passion of games and how they work since I was a kid. Growing up spending hours at net cafes (playing CS 1.5, WC3, D2), later getting into lanning with mates, and then running and helping various game servers for small communities - all of which lead to my role at GGG, where I first started in Support then moved to QA.

Lionel: Hello, I'm Lionel. I like e-sports, so ultimately the manifestation of pure skill in competition. Super Smash Bros Melee is the best game of all time. Recently, there were a few speedrunners trying out Path of Exile, and it was really interesting to read their perspective. I haven't watched as many tournaments in the past few weeks, and it was refreshing to hear about the small details that make them proficient in Diablo II and their love for the game as it resonated with aspects of my relation to video games and what makes me enjoy them the most.

Could you please tell us about your current role at Grinding Gear Games and what an average work day is like for you?

Kieren: What I enjoy about QA is you look at something new every day. The first thing once we get in is finding out whether any patches are main priority, which involves QA running through checklists for each feature, testing them again on a local server with the patch, and informing others if there's any problems and fixes needed before they're deployed.

If we're not focusing on a patch, then we're mainly going through a list of resolved tickets consisting of bugs that have been worked on, or features added by other staff - quite interesting since we get to see or talk to them about what they've been doing. These issues get checked on our main testing server, and are re-opened or closed depending whether we find any problems or have suggestions.

We also keep a close eye on bug reports as well crash information we receive, attempt to reproduce them, and send tickets with all the information and reproduction steps (and gifs/videos) to someone who can look into fixing it - if there isn't a ticket already. The same steps are used if we find any other bugs while playtesting, and some of us have spreadsheets to keep track of the progress for large projects/testing that involves many tickets.

Lionel: Yeah what he said, except the first thing we actually do is pick up Blue V from the fridge.

Kieren: The vanilla coke is where it's at right now though, you have to get in quick before they all disappear for the day.

Please, tell us about Quality Assurance at GGG. How many people are in your QA team?

Kieren: Currently there are 9 of us in the team, we all get along well and learn from each other. Since we're all checking work from other roles in the company, keeping each other informed and helping one another is quite important.

Do you playtest manually or do you use specific software/tools/bots?

Lionel: We don't use bots, however we have tools which can be a mix of ahk scripts usually done by QA or cheats implemented by programmers. Rian touched on adding some during his interview, and he saved us hundred of hours of tedious mindless tasks. The best one is called /dad, I'll let the community figure out what it does.

Playtesting happens, however for most of our duties it isn't the most relevant/efficient approach, and it's rightfully perceived as a luxury that can only happen when things are going smoothly. This is typically not the atmosphere in the weeks leading to a major patch.

Kieren: As Lionel's said - we have a lot of tools and scripts we make, as well as cheats and that a few of our programmers come out with. These are hugely helpful, and certainly essential to much of what we do: from when changes are made to lists of items, mods, or monsters; cheats for testing new league mechanics; and for when checking characters at different stages of playthrough.

One big thing that's happened while I've been here is being able to paste lines of commands into the client, and have it perform them in sequence - which is great for being able to reproduce bugs, pass the steps on to someone so they can look at it more in depth, and then test the same case again later to see if it's fixed. For example, you might have a situation where you create a character with specific uniques and passives, need to warp to a certain area, warp to a monster, and then use specific skills. This could turn out to be like 40 lines of commands but rather than doing them one by one each time, we can just paste it all in at once to have the scenario ready.

Could you describe the process of testing, please? How many times do you test the same content? Do you test it during all of its development stages, or do you test only final version?

Lionel: Let's take a boss fight: more often than not, we start with an incomplete prototype. Things like models, animations, spell effects, audio are in various states but we can look at skills in terms of general behaviour and try to think on the cases where they would break. There is frequently a back and forth between programmers, designers and a small part of QA to determine how skills evolve from the original idea which is in a document, and how its implementation will slightly differ.

The amount of time we test the same content depends on how broken it is, and how quickly problems get resolved as well as the time available. One of the upcoming boss is using a beam ability, and to get it right it took easily a dozen of tickets as fixing each problem kept introducing new ones. On the other hand, the breach-spawning technology was clear right from the start (outside of the monsters being invisible on the edge which was something that couldn't be fixed), and despite trying out many ways to break it, that feature was always correct. Now I hope I'm not forgetting a small bugfix, but if I do I'm sure the community will kindly remind us.

Different people are involved in different stages, so depending on who you ask you'll get a different answer. It is always interesting to hear someone's else perspective from a new angle, as it can bring a light on a detail that isn't quite right.

Kieren: We also have checklists we run through to try cover everything: microtransactions, monsters, uniques, and quest throughs, although tickets for features requiring more work are often split up. For instance we'd be looking at monsters' skills in separate tickets, their audio in another, then the monster is looked at when it's in the area, and checked again before it's patched to the live realm. Features at the last step are often checked by multiple QA too. It's quite a lot of checks, but due to the nature of the game and the mechanics behind it all, changes made can affect very specific situations as well as things that aren't as obviously related.

Can you influence the development of Path of Exile? Have you ever had conflicts with designers or programmers regarding balance or mechanics?

Kieren: Oh yeah, we tell them to make the game as easy as possible for everybo- kidding… Discussions do happen with QA and other staff when they're after feedback or brainstorming ideas, if there's any conflicts then they tend to be taken on board and weighed up fairly between those involved. There's also suggestions always being added to tickets by us that are considered and sometimes added to the game, sometimes not.

Do you play Path of Exile when you have free time? How do you keep it interesting after playing Path of Exile all day at work?

Lionel: Playing Path of Exile all day at work is a misconception. Very little of our daytime is actually spent playing the game in a way players do. I've gotten far worse as a player since I started working as QA. While I don't have the same drive to play the game as I used to, I am looking forward to the Fall of Oriath expansion and I hope more consideration is on the way for the competitive aspects of the game.

Kieren: Yep! Being part of, and seeing the creation of the new content tends to make it more enjoyable to experience (while reminding yourself you're away from a testing environment). Although you also look forward to much of what's to come, and there's always something new around the corner here.

Where do you usually find bugs? Do you monitor the forum or Reddit, or do you find them by yourself?

Lionel: The bugs we find on reddit/bug reports are the tip of the iceberg, not that I mean to devalue their importance, quantity or game breaking aspects. I've been on the receiving end of broken content, so I can understand the frustration players feel, however constantly reading that "GGG has no QA" can be discouraging. Path of Exile is a very complex game with parts that interact with each other in unexpected ways, and it's very hard to think of all the possible interactions.

For an expansion, we usually have half the QA department present on launch day, and monitoring the bug report section, reddit and global chat to quickly work out bugs and send them to programmers. Kieren spoke about this process in the "what's your daily day like", but launch day can be more frenetic.

Kieren: Well, we find and have bugs fixed everyday with things that are still being worked on. As for bugs on the live realm, after scouring the community message boards we pass on tickets for any problems we can reproduce. Having a good amount of detail on how we can check the problem in these reports helps us heaps, and then we're also pretty quick to update the Known Issues thread stickied in the Bug Reports forum. Following that we're sometimes asked to assist in reproducing common crashes that are reported back to us, or just notice occasional problems while playing or checking something unrelated.

Could you tell us what qualifications are required to become a QA tester at GGG?

Kieren: An enjoyment of the game, a fair amount of knowledge about it. A good eye for detail, and being able to look at things from different perspectives also helps for giving feedback. Living in the same city is probably a must, on account of needing to come into the office and all.

Can you remember any funny bugs?

Lionel: No. Bugs are funny until you see them ruining someone's gameplay experience.

Kieren: Ah, we see so many. There was the time the shield charge effects persisted when you were knocked back mid-charge:



Another one comes to mind where the game wasn't quite looking right on my computer (elemental testing weapon unrelated):



The Passive Tree looked great:



And if you scrolled in a little it looked like your computer was dying:



Thankfully that one got fixed before it went live.