Hello everyone! I’m Sitri, a Community Intern here at Warner Bros. Games Boston, and welcome to the seventh and final installment of our Developer Interview Series.
I can’t believe it’s been over half a year since the start of the series! Time flies when you’re crash coursing game development.
For the finale we’ve decided to go big, interviewing three different departments to see how they work in tandem as cornerstones of player advocacy and experience: Quality Assurance (QA), Consumer Insights (CI), and Customer Service (CS).
At first glance, you might not see the connection between these departments. They exist in different corners of our organization; however, they synergize in working towards similar goals – listening to feedback and using that to help players, flag issues, and influence development.
They each approach this challenge differently though, so let’s introduce the team members and hear more about it from them!
Q: Could you provide a brief introduction on your role in the studio?
Merlina: As a Consumer Insights Analyst, I conduct research to ensure that the player experience is at the forefront of game design and decision-making. To put it simply, I write the surveys that get sent to players and then I present the findings to the Game of Thrones: Conquest team and make recommendations. If you’ve ever filled out a Game of Thrones: Conquest survey, chances are that I wrote it! I also occasionally get the chance to work on qualitative usability and UX research for new features, such as Heroes and Battlegrounds.
Sunny: I’m the middleman between the game team and the Customer Service Agents. When something goes wrong in the game, our Agents are usually the first to hear about it. I report it to the game team, we discuss our plans to resolve the issue, and relay that to the agents. I’m also involved with the implementation of new features to assure that it’s a smooth player experience from the start.
QArth: I am the manager for our Quality Assurance team. If there’s something that needs testing, my team is ready and available to test it thoroughly. I have team members embedded in all development teams in the studio as well as flexible testers who can help when needed. I am accountable for making sure each bug is identified as early in the development process as possible.
Q: How would you define your discipline and the work you do?
Merlina: Consumer Insights is a research-driven role that analyzes player perceptions and behavior in order to advise and inform the game team so they can continually iterate and improve upon their work. I don’t make the game, but I help make the game better.
Sunny: Customer Service is a role that most jobs have some sort of parallel with. It’s a discipline that most people will have at least a little experience in, but that experience will vary differently from person to person. In my experience, I would define Customer Service as a tool for players to keep their gameplay as smooth as possible, and a tool for the development team to better understand our players and the issues they experience.
QArth: The QA team is responsible for testing and validating all the features and events that are in the game. If there is a bug located during development, the QA team will file it for our developers and designers to fix. QA is one of the more important checkpoints for any game change going out to players. The QA team also responds to issues that may emerge after a feature is shipped, records bugs, and updates test plans as needed to prevent future issues.
Q: What’s your “day in the life” for Game of Thrones: Conquest?
Merlina: Typically, a team within the studio will approach me with questions that they have about players. It could be player interest in a new feature they want to implement, the perceived value of a key game mechanic, or reactions to an event that was just run for the first time. I put together a research plan, conduct the research, analyze the results, and finally present the findings to the team. Prioritization of projects largely has to do with studio priorities and time-sensitive topics, such as a scheduled event.
Sunny: My day can vary quite a bit every day! It all depends on what we have planned for the game, and what goes wrong. My first priority is always to tackle any issues as they arise. Once we’re in the clear, I act as a consultant for the other teams on behalf of the players.
QArth: The day-to-day for me can be anything from attending planning meetings, organizing testing estimates, reviewing test plans, and reviewing proper coverage for each development group. Each QA team member has a more focused day-to-day, providing feedback and ensuring test coverage for all features.
QA Analysts build test plans in preparation for the project, and then execute the test plans as work is made available by an Engineer or Designer. Once the work has been validated as functional, it is approved for shipment. If a QA member locates a bug, they will block that work from making its way to the players.
Q: How are our players important to the work you do?
Sunny: Extremely. As I mentioned, I always try to act on their behalf and work on making the game more accessible and enjoyable for them.
Merlina: Players are my whole job! I couldn’t do any part of my job without talking to the players, getting their feedback, hearing their concerns, and making their voices known to my colleagues who make this game.
QArth: QA is involved with every facet of Game of Thrones: Conquest and has reviewed the entire game and all new features since before the game was available in the app stores. With that said, players see all the work that QA does here at the studio. I like to think of QA as the “gatekeepers” for all code going out the door. If for any reason a bug is still located in the live game, we will investigate, get the issue filed up to get fixed, and record how we can better test in the future. For every bug a player does unfortunately see, there were 100 more that QA had filed and verified before release.
Q: How do you collaborate with different departments and teams in the studio?
Sunny: CI and QA are two of the most important departments that I interact with. We’re all exchanging information with each other and using that information to drive future decisions.
Merlina: I work with any department and team that has questions about players. A lot of the time, that means the UX, Design, and Product teams. I often work with Customer Service and Community to collaborate and pull in our varying data points to create a larger picture of the player experience. I also work with the awesome folks in QA to help me set up and run user tests on unreleased or in-progress features.
QArth: QA interacts with nearly every department here at the studio. Community, CS, and QA collaborate closely to make sure issues are prioritized correctly and that we are hearing player concerns. We work with Engineering, Design, Art, UI/UX, and more to understand how a feature should work (and shouldn’t work!) so that detailed test plans can be created. We work closely with our production department to make sure bugs are being incorporated into each workflow to be fixed in priority order. Essentially, we’re always listening for new information to improve testing efficiency and coverage.
Q: What are some common misconceptions about what you do?
Merlina: I think that some players believe that their one survey response can’t make a difference or that we don’t read the write-in responses to essay questions, but we do. In fact, every report that I present to the Game of Thrones: Conquest team has player-written feedback pulled directly from survey responses as evidence of a change that should be made, a concern that players have, or a particularly positive reaction to a recent addition.
Sunny: I think many people may not realize that my job exists at all. Games have evolved radically over time, but not always in the way we talk about them. Customer Service is a large arm in game development, especially for live service games.
QArth: It is a common misconception that every bug in the game will get fixed at the same rate. QA locates as many bugs as possible with the highest confidence in a reasonable amount of time. Once identified, the bugs must be prioritized and placed into a stack rank of importance for fixing. This is based on many factors such as impact to players, impact to gameplay, the complexity of the issue, and much more. The higher priority bugs get fixed first, leading the lower priority bugs to wait their turn.
Q: How can players best word feedback to make it as actionable as possible?
Merlin: It’s important to remember when filling out a survey to always answer based on your own experiences or opinions. Many players will write something like “I don’t like XYZ, but other players might,” which is hard to draw conclusions from. When it comes to write-in responses, the most important information is the why. It’s good to know that you like or dislike something, but it’s even more valuable to know why you like or dislike it so we can make more positive changes. It’s silly, but I once got a comment on a survey thanking me for providing them an outlet for feedback, which just gave me the warm and fuzzies.
Sunny: Be specific! If you experience a bug in the middle of battle that you’d like to inform CS about, then let us know what you were battling (player, NPC, event creature) and the results you expected versus what actually happened. The more information the better! But, most importantly, please have the feedback be nice (or at least not mean). We understand that bugs can be frustrating, but we’re just people trying to help!
QArth: I will echo the above statements. When you experience an issue in the game, the more information that you can give to us the better. Specifically with a bug, what helps the most is knowing the exact actions taken to cause the bad behavior. These could be as simple as “1. Open the game, 2. Click button X, 3. Check for Y”. Even better is if these steps are consistent and repeatable, then we will be able to reproduce the issue on our end and be better equipped to prioritize a fix. Additionally, taking videos and screenshots of the issue are invaluable to my team when it comes time to verify the issue has been resolved.
…. And now, some fun facts about the team!
Q: Who’s your favorite Game of Thrones character?
Merlina: Call me a cliche, but I have a huge crush on Jon Snow. He is a precious boy with a precious wolf and he can do no wrong.
QArth: Arya is definitely my favorite character in the series. She is a badass who didn’t take anyone’s nonsense. She also basically saved the entire continent, so that’s cool too.
Sunny: I’ll definitely have to agree with QArth on Arya!
Q: If you were a stat, what would you be?
Merlina: Well, my job is research, so I guess I would be a research stat. I’m also obsessed with Dragons, so I’ll say Dragon Research Speed. Get me a powerful dragon as soon as possible please.
Sunny: Definitely Healing Speed. When something breaks, I have to act fast to get it fixed.
QArth: Free Build Time. I love building my Keep and anytime I can reduce the timer to build faster is awesome.
Q: What would your house emblem/animal/heraldry be if you were a great House of Westeros? Bonus points if you include your House words too!
Merlina: Oh, a dragon for sure. Although my sigil in the game is hedgehogs, they might not convey the same sense of power and intimidation you would need to survive in the brutal and unforgiving lands of Westeros.
Sunny: I love the chicken emblem with the dragon tail. Our incredible Art team somehow made them look so cool. It’s the perfect balance of hardcore and funny.
QArth: I find the badger, particularly the honey badger, to be a very cool animal. They are tough, brave, and can’t be stopped. Therefore, my emblem is of a badger.
We hope you enjoyed this look into our Consumer Insights, Customer Support, and Quality Assurance teams! We’ve come a long way from starting this series nearly a year ago, but it feels appropriate to be ending with these disciplines that interface so directly with our community.
Joining the studio as an intern, I had so many questions about how I fit within its structure and how the wider world of game development operated. I am grateful for the opportunity to take our community along for the ride, so we can all learn more about just how much goes into making a game like Game of Thrones: Conquest.
Thank you to everyone who tagged along this journey with me and chimed in, whether it be on discord or social media, with questions and comments throughout the series.
This series may be coming to an end, but we are by no means done sharing the stories of our studio! There are so many other disciplines that play a crucial role in the making of Game of Thrones: Conquest such as Data Analytics, Product Management, and Publishing, to name a few.
The story continues on our official Discord community- If there is any area we missed that you are really interested in hearing from, join the conversation and let us know!
You can read our previous posts under our Developer Interview tag, or in the new #dev-archive Discord channel in the coming weeks.