Development Automation: How to Prioritize + Best Practices

Oculus Developer Blog
|
Posted by Oculus VR
|
December 17, 2020
|
Share

While automation is common throughout most types of software engineering, it has yet to become a common practice for VR app and game development. We want to help you get started with using automation, or at a minimum, understand how to assess, get started, and scale your automation and testing efforts.

This post consists of a series of recommendations and best practices to keep in mind as you prioritize and implement automation. It is an excerpt from a larger guide we recently published in a series focusing on Minimizing Iteration Time and Maximizing Developer Efficiency. The larger guide includes sections on General Considerations and Cross Platform Development, Project Planning and File/Folder Management, Automation and Performance Optimization, as well as Best Practices specifically for Unity development. Check out each of these guides as they include insights gathered from interviews with cross functional VR subject matter experts throughout Facebook Reality Labs.

For those new to software automation, we especially recommend this guide on automation, as it includes a primer on the different testing scenarios that will help you prioritize where you focus your automation resources.

How and when to prioritize automation

Automating the test process can help you a great deal, especially in the long term, but there are a number of items to consider before you begin to focus on automation script writing and the technical requirements that go along with it. See below for a few key factors you should keep in mind as you design your testing and automation strategy:

Consider time and skill

Be prepared to analyze the cost benefit analysis of writing a rock solid test script as opposed to using manual testing. The more complex the test, the more time and effort will be required to write and maintain.

All things considered, automation is recommended

With the above point in mind, if you have the ability to automate any of your processes, whether something as small as a boot test, a complex interaction, or full environment, automation can help your team drastically speed up iteration time and minimize the number of bugs/issues that arise in your app over time.

Automate elements that are repeatable and focused

Automate sections of your app that are repeatable and generally less complex. This will maximize efficiency and ensure your script accurately delivers the designed outcome.

Target edge case opportunities

Employ automation in those strange instances where it would require a great deal of logistical work for a single, manual test, or user flows that might be considered “abnormal.”

Testing and automation best practices

After you’ve decided what type of tests you’re looking to automate and where to focus your resources, use the following best practices to move forward with your test automation process.

Test scripts as inspiration to automate

As your team most likely already uses test scripts to drive your QA process, these can be an easy way to find those repeatable areas of your build that could be automated with minimal effort.

Use source control and continuous integration

If you are looking to publish a fully developed Oculus VR app, it’s recommended that you leverage a source control and continuous integration system. These come in many different shapes and sizes, and it’s recommended you scale your system with the size of your team and type of app you’re creating. For source control systems we recommend taking a look at Git and Mercurial, or Perforce if you have a more complex app. For continuous integration systems check out Jenkins, TeamCity, or GitLab.

2D version of your app recommended

Whether you’re using test automation or not, be sure to include a version of your app that can simply be run on a 2D screen. This will save you plenty of time for testing certain features, variables, and properties, and speed up your debugging process.

Leverage telemetry when possible

This sort of data can be extremely powerful in the long term as you work to analyze what may have caused a specific error. When planning on what telemetry data to log, think about the user journey through your app and the user flows that need to be completed for a successful playthrough.

Detect build errors early

Compiling code and running the asset build on your CI system can detect issues as soon as a new change is committed. Proactively warning your team of these errors can ensure that they avoid any conflicts as well as steer clear of any potential blockers these issues might cause.

Highly recommended for performance testing

On-device automation can be especially helpful for monitoring performance. New changes can unintentionally harm performance, and detecting this as soon as possible will be much easier to fix when the cause is known. Monitoring performance over time can also ensure that your updates to optimize performance are effective. See below for more information on testing your app’s performance.

Trigger events on device

Trigger specific events on the VR headset, for example “tell character to go to X world and move character to Y position”. Examine performance characteristics over ADB and be sure to run certain checks throughout the event.

More VR Development guides on automation, performance optimization, and ways to minimize iteration time

If you’re looking to learn even more about VR development automation, performance optimization, and automating your testing process be sure to check out the links below.

And if you haven’t already visited the Learn section of our developer center, here are a few recently updated guides for you to further improve your processes and elevate your skillset.

Feel free to leave any comments or questions in the comments section below, and keep up the great work!