Get 5 fantastic insights for betting your automated tests and related processes! I’m Alexey Fedotkin, and I focus on Automated Testing at Pixonic (MY.GAMES). In this post, I’d like to share valuable insights that my team and I have gained over the past few years while developing automated tests for the War Robots project. Quick disclaimer: since the effectiveness of different approaches and techniques often depends on the specific development context, the project itself, and the team involved, our experience is particularly relevant to similar projects (especially mobile games and shooters). That said, many of these insights are universal and can benefit everyone. Tip 1: Emphasize Transparency An often-overlooked principle is that "any business process should be transparent and understandable." But why is this important? This is because ambiguity breeds questions and mistrust, which can lead to wasted time and resources. Furthermore, this can result in negative feedback about your engineers’ work – potentially creating more significant problems down the line. For instance, we’ve hosted several meetups where we discussed our testing processes and capabilities. As a result, we noticed a significant increase in review requests for various feature branches coming directly from the development team. Why? This shift occurred because our colleagues understood that automated tests aren’t just some abstract concept handled by a few people in the studio. Rather, they’re valuable tools that can eliminate errors and streamline future merges. After this, we received ideas for test improvements and refinements, focusing the AQA team’s efforts on what the project genuinely needs. Ultimately, this type of collaboration makes it easier for everyone to deliver high-quality content to production. Tip 2: Ask Questions While this point leans more towards traditional QA than automation, it's crucial to continually ask questions about what users want and what gaps exist. People often won't voice their needs or the tools that could simplify their tasks unless prompted directly. As an example of this, we utilize large tables to describe game balances and these contain significant amounts of data. Occasionally, errors can crop up within these tables since they’re handled by people, and errors are also a natural part of the process. After discussing things with the game designers, we identified the specific rules the data must adhere to, the associated limitations, and, crucially, the need for data verification – something which we often lack the necessary resources to do. Yet, this oversight contributes to periodic errors and inaccuracies in production data. And while these issues can be resolved quickly without requiring fixes in the game client or a full review cycle, they can still impact player experience and perceptions of the game. To solve this, we created a script that scans all of the relevant tables for inconsistencies against established rules. This allows us to identify and correct errors before rolling out the balance updates to production. Tip 3: Set Priorities Writing tests is a resource-intensive and time-consuming endeavor. Moreover, software development is rife with unexpected challenges — ranging from environmental issues to shifts in team priorities regarding product direction. Thus, it's essential to keep clarifying which content is most critical, those that should be prioritized for testing, and those which will deliver the greatest value to the product. To demonstrate a case of this in the wild, we developed a prioritization map for content coverage. This map allows the entire team to easily access Confluence and view which elements will definitely be included in the automated regression testing for the upcoming release, as well as those which may not make it and will require closer attention from the test engineers. Aim to test game mechanics rather than the content itself. This approach allows for broader coverage while minimizing testing time. Normal test: [TestCase(Equip.Scald)]
[CustomRetry]
[Description("Verify that Blast has a cumulative effect and is applied after filling the corresponding scale")]
public void Blast_Cumulative_Effect_Test(Equip blastEquip)
{
   PrepareCoreEnvironmentHelper
       .SetLevelAndSkipPopups() “Overkill” approach: [TestCase(Equip.Scald)]
[TestCase(Equip.Daeja)]
[TestCase(Equip.Exodus)]
[TestCase(Equip.Quarker)]
[TestCase(Equip.Vengeance)]
[CustomRetry]
[Description("Verify that Blast has a cumulative effect and is applied after filling the corresponding scale")]
public void Blast_Cumulative_Effect_Test(Equip blastEquip)
{
   PrepareCoreEnvironmentHelper
       .SetLevelAndSkipPopups() Case in point, consider five rocket launchers that each fire single shots and apply “Effect A” upon impact. The mechanics are defined in the code only once and remain consistent; only the balance numbers and visual skins change (accuracy in these aspects is verified at a different level, not through functional tests). Therefore, a single test can effectively verify the functionality of the code that describes Effect A and the application of damage numbers from the balance for the rocket launcher. As the number of tests grows, efficiency becomes critical — even when you have a robust infrastructure for executing them. Tip 4: Maintain Your Test Farm It's important to monitor your testing environment closely. If you work remotely and are not often in the office, consider setting up a camera to stream the testing process in real time. This will facilitate easier analysis of tests and help you quickly identify the causes of any failures. Additionally, create utility scripts for routine tasks like restarting the system, clearing space, or checking device statuses. These tools will extend the lifespan of your equipment and reduce failures, resulting in fewer test restarts and less wasted time. Tip 5: Continuously Enhance Your Code Here’s an undisputed truth: you should always prioritize maintenance and improvement of your test code. Regardless of how diligently you write your code or how carefully you avoid design flaws, errors will inevitably occur. Mistakes are part of the human experience. However, so is the ability to rectify them – whether they pertain to code or the testing process! After all, in essence, any test case is a set of steps to get from point A to point B. As your number of tests grows, so will the frequency of these "paths" through the screens. Don’t hesitate to streamline them, including seeking assistance from developers in the process. Request updates to old tests, acquire new cheats, and adjust your test cases to reflect the current state of the project. Continuous improvement in this area will always yield benefits. For instance, in our game, we have drones that serve as support units for robots. Some of these drones have mechanics that activate upon dealing specific damage values within a certain timeframe. Previously, the testing process required us to select the appropriate weapons, set shooting durations, and define error margins. Now, we’ve developed a cheat that can apply the required damage from one specified mech to another. Now it looks like this: Tools.BattleActionsHelper.ApplyDamageToSpecifiedMech(ToolsPack, Players.SecondPlayer, Players.FirstPlayer, EquipSlotsList.FirstSlot, damageAmount); Works literally in one and a half seconds, does not depend on rebalances and the speed of the particles from the weapon And here's how it was before refactoring: ToolsPack[DroneAbilityHelper.SecondPlayer].CommonActions.PressSelectedElementDuringTimeInterval(BattleUiKeys.FireAllWeaponsFor4ThSlotButton, ShootingTimeMs); You need to shoot, but before that you need to think about all the guns or from a part, you need to remember about the distance and dispersion when shooting, you need to wait until the required number of shots hit the target. If there is a rebalance, this can make the tests unstable... in general, everything was difficult Is it cool? Absolutely! Is it efficient? Definitely! So, why didn't we implement this sooner? The answer is simple: there was no demand for it. That said, as the project has evolved, this need has emerged, making its implementation relevant now. Inevitably, you’ll encounter challenges and obstacles with your tests. It's important to remember that there’s no one-size-fits-all solution, so achieving perfection is unlikely. However, this doesn't mean you can't address the issues at hand — especially since many of these hurdles have already been navigated by industry colleagues whose experiences can provide valuable insights! Get 5 fantastic insights for betting your automated tests and related processes! Get 5 fantastic insights for betting your automated tests and related processes! I’m Alexey Fedotkin, and I focus on Automated Testing at Pixonic (MY.GAMES). In this post, I’d like to share valuable insights that my team and I have gained over the past few years while developing automated tests for the War Robots project. Quick disclaimer: since the effectiveness of different approaches and techniques often depends on the specific development context, the project itself, and the team involved, our experience is particularly relevant to similar projects (especially mobile games and shooters). That said, many of these insights are universal and can benefit everyone. Tip 1: Emphasize Transparency An often-overlooked principle is that "any business process should be transparent and understandable." But why is this important? This is because ambiguity breeds questions and mistrust, which can lead to wasted time and resources. Furthermore, this can result in negative feedback about your engineers’ work – potentially creating more significant problems down the line. For instance, we’ve hosted several meetups where we discussed our testing processes and capabilities. As a result, we noticed a significant increase in review requests for various feature branches coming directly from the development team. Why? This shift occurred because our colleagues understood that automated tests aren’t just some abstract concept handled by a few people in the studio. Rather, they’re valuable tools that can eliminate errors and streamline future merges. After this, we received ideas for test improvements and refinements, focusing the AQA team’s efforts on what the project genuinely needs. Ultimately, this type of collaboration makes it easier for everyone to deliver high-quality content to production. Tip 2: Ask Questions While this point leans more towards traditional QA than automation, it's crucial to continually ask questions about what users want and what gaps exist. People often won't voice their needs or the tools that could simplify their tasks unless prompted directly. As an example of this, we utilize large tables to describe game balances and these contain significant amounts of data. Occasionally, errors can crop up within these tables since they’re handled by people, and errors are also a natural part of the process. After discussing things with the game designers, we identified the specific rules the data must adhere to, the associated limitations, and, crucially, the need for data verification – something which we often lack the necessary resources to do. Yet, this oversight contributes to periodic errors and inaccuracies in production data. And while these issues can be resolved quickly without requiring fixes in the game client or a full review cycle, they can still impact player experience and perceptions of the game. To solve this, we created a script that scans all of the relevant tables for inconsistencies against established rules. This allows us to identify and correct errors before rolling out the balance updates to production. Tip 3: Set Priorities Tip 3: Set Priorities Writing tests is a resource-intensive and time-consuming endeavor. Moreover, software development is rife with unexpected challenges — ranging from environmental issues to shifts in team priorities regarding product direction. Thus, it's essential to keep clarifying which content is most critical, those that should be prioritized for testing, and those which will deliver the greatest value to the product. To demonstrate a case of this in the wild, we developed a prioritization map for content coverage. This map allows the entire team to easily access Confluence and view which elements will definitely be included in the automated regression testing for the upcoming release, as well as those which may not make it and will require closer attention from the test engineers. Aim to test game mechanics rather than the content itself. This approach allows for broader coverage while minimizing testing time. Normal test: Normal test: [TestCase(Equip.Scald)]
[CustomRetry]
[Description("Verify that Blast has a cumulative effect and is applied after filling the corresponding scale")]
public void Blast_Cumulative_Effect_Test(Equip blastEquip)
{
   PrepareCoreEnvironmentHelper
       .SetLevelAndSkipPopups() [TestCase(Equip.Scald)]
[CustomRetry]
[Description("Verify that Blast has a cumulative effect and is applied after filling the corresponding scale")]
public void Blast_Cumulative_Effect_Test(Equip blastEquip)
{
   PrepareCoreEnvironmentHelper
       .SetLevelAndSkipPopups() “Overkill” approach: “Overkill” approach: [TestCase(Equip.Scald)]
[TestCase(Equip.Daeja)]
[TestCase(Equip.Exodus)]
[TestCase(Equip.Quarker)]
[TestCase(Equip.Vengeance)]
[CustomRetry]
[Description("Verify that Blast has a cumulative effect and is applied after filling the corresponding scale")]
public void Blast_Cumulative_Effect_Test(Equip blastEquip)
{
   PrepareCoreEnvironmentHelper
       .SetLevelAndSkipPopups() [TestCase(Equip.Scald)]
[TestCase(Equip.Daeja)]
[TestCase(Equip.Exodus)]
[TestCase(Equip.Quarker)]
[TestCase(Equip.Vengeance)]
[CustomRetry]
[Description("Verify that Blast has a cumulative effect and is applied after filling the corresponding scale")]
public void Blast_Cumulative_Effect_Test(Equip blastEquip)
{
   PrepareCoreEnvironmentHelper
       .SetLevelAndSkipPopups() Case in point, consider five rocket launchers that each fire single shots and apply “Effect A” upon impact. The mechanics are defined in the code only once and remain consistent; only the balance numbers and visual skins change (accuracy in these aspects is verified at a different level, not through functional tests). Therefore, a single test can effectively verify the functionality of the code that describes Effect A and the application of damage numbers from the balance for the rocket launcher. As the number of tests grows, efficiency becomes critical — even when you have a robust infrastructure for executing them. Tip 4: Maintain Your Test Farm Tip 4: Maintain Your Test Farm It's important to monitor your testing environment closely. If you work remotely and are not often in the office, consider setting up a camera to stream the testing process in real time. This will facilitate easier analysis of tests and help you quickly identify the causes of any failures. Additionally, create utility scripts for routine tasks like restarting the system, clearing space, or checking device statuses. These tools will extend the lifespan of your equipment and reduce failures, resulting in fewer test restarts and less wasted time. Tip 5: Continuously Enhance Your Code Here’s an undisputed truth: you should always prioritize maintenance and improvement of your test code. Regardless of how diligently you write your code or how carefully you avoid design flaws, errors will inevitably occur. Mistakes are part of the human experience. However, so is the ability to rectify them – whether they pertain to code or the testing process! After all, in essence, any test case is a set of steps to get from point A to point B. As your number of tests grows, so will the frequency of these "paths" through the screens. Don’t hesitate to streamline them, including seeking assistance from developers in the process. Request updates to old tests, acquire new cheats, and adjust your test cases to reflect the current state of the project. Continuous improvement in this area will always yield benefits. For instance, in our game, we have drones that serve as support units for robots. Some of these drones have mechanics that activate upon dealing specific damage values within a certain timeframe. Previously, the testing process required us to select the appropriate weapons, set shooting durations, and define error margins. Now, we’ve developed a cheat that can apply the required damage from one specified mech to another. Now it looks like this: Tools.BattleActionsHelper.ApplyDamageToSpecifiedMech(ToolsPack, Players.SecondPlayer, Players.FirstPlayer, EquipSlotsList.FirstSlot, damageAmount); Tools.BattleActionsHelper.ApplyDamageToSpecifiedMech(ToolsPack, Players.SecondPlayer, Players.FirstPlayer, EquipSlotsList.FirstSlot, damageAmount); Works literally in one and a half seconds, does not depend on rebalances and the speed of the particles from the weapon Works literally in one and a half seconds, does not depend on rebalances and the speed of the particles from the weapon And here's how it was before refactoring: ToolsPack[DroneAbilityHelper.SecondPlayer].CommonActions.PressSelectedElementDuringTimeInterval(BattleUiKeys.FireAllWeaponsFor4ThSlotButton, ShootingTimeMs); ToolsPack[DroneAbilityHelper.SecondPlayer].CommonActions.PressSelectedElementDuringTimeInterval(BattleUiKeys.FireAllWeaponsFor4ThSlotButton, ShootingTimeMs); You need to shoot, but before that you need to think about all the guns or from a part, you need to remember about the distance and dispersion when shooting, you need to wait until the required number of shots hit the target. If there is a rebalance, this can make the tests unstable... in general, everything was difficult You need to shoot, but before that you need to think about all the guns or from a part, you need to remember about the distance and dispersion when shooting, you need to wait until the required number of shots hit the target. If there is a rebalance, this can make the tests unstable... in general, everything was difficult Is it cool? Absolutely! Is it efficient? Definitely! So, why didn't we implement this sooner? The answer is simple: there was no demand for it. That said, as the project has evolved, this need has emerged, making its implementation relevant now. Inevitably, you’ll encounter challenges and obstacles with your tests. It's important to remember that there’s no one-size-fits-all solution, so achieving perfection is unlikely. However, this doesn't mean you can't address the issues at hand — especially since many of these hurdles have already been navigated by industry colleagues whose experiences can provide valuable insights!

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

Read My Stories

Too Long; Didn't Read

Make resilience your competitive advantage

A Game Developer's Guide to Making Your Automated Tests Awesome in 5 Simple Steps

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

11 Best Automation Testing Tools to Try in 2021

145 Stories To Learn About Test Automation

Python-Powered Performance Testing for QA Testers: A Beginner's Guide to Cloud API Load Testing

57 Stories To Learn About Automated Testing

7 Steps to Effectively Implement Test Automation for Video Games

11 Best Automation Testing Tools to Try in 2021

145 Stories To Learn About Test Automation

Python-Powered Performance Testing for QA Testers: A Beginner's Guide to Cloud API Load Testing

57 Stories To Learn About Automated Testing

7 Steps to Effectively Implement Test Automation for Video Games

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps