Meta Developers

You can run automation tests for Unity and Unreal Engine projects with Meta XR Simulator. This introduces a new way to test many manual tests, particularly smoke tests and regression tests. This is a zero-code, record-and-replay approach utilizing “snapshots” to verify test results.

These are the steps to set up and run automated tests for your game:

Record tests.
Replay and generate the expected test result.
Transfer your game and test files to your test machine and run the test.
Compare test results.

Prepare test

Tests can be recorded on a developer’s local computer.

Record tests

Install and set up the Meta XR Simulator with your development environment. For more info, see Getting Started with Meta XR Simulator.
Run your game in Meta XR Simulator.
Click Session Capture > Record to start a recording.
Select a VRS file to record to (for example, recording.vrs):
- Notice that Session Capture is now in the Recording state
Use a mouse and keyboard (or a connected controller) to interact with your game.
Take snapshots by pressing Take Snapshot(s) at key spots. These will later be used to verify the test result.
When you are done recording, press the Stop Recording button.
Repeat the above steps to make a new recording for other test cases. Each test case is recorded and saved into a VRS file.
- Replay recordings to generate the expected test result

Replay recordings to generate the expected test result

Run your game again on Meta XR Simulator.
Start a replay by clicking Session Capture > Replay.
In the prompt window, select a Recording VRS file to replay.
In the second prompt window, select a VRS file to save the replay (for example, replay_expected.vrs).
Meta XR Simulator will automatically run the recording.
Once done, press Session Capture > Stop Replay.
Repeat the above steps to generate a replay VRS file for each recording. The replay.vrs file contains the expected test result – screenshots.

Execute automation tests

The following steps demonstrate how to execute a recorded test from the previous section. While these steps can be automated with scripts, this document outlines them step-by-step so you can customize and align them with your CI process.

Transfer the recording.vrs and replay.vrs files to the test machine along with your game

Add the following JSON code block into %AppData%\Roaming\MetaXR\MetaXrSimulator\persistent_data.json.

 "session_capture": {

 "delay_start_ms": 1000,

 "exec_state": "replay",

 "quit_buffer_ms": 1000,

 "quit_when_complete": true,

 "record_path": "C:\\open\\test_recordings\\recording.vrs",

 "replay_path": "C:\\open\\test_recordings\\replay_new.vrs"

 }

Setup the Meta XR Simulator with your development environment. For more info, see Getting Started with Meta XR Simulator. A. Visit this page to find the direct download for the latest Meta XR Simulator B. Only the MetaXrSimulator folder is needed. Copy it to somewhere convenient, like C:\Projects\MetaXRSimulator (this location will be assumed for the rest of this document). C. Run the Synthetic Environment server: cd C:\Projects\MetaXRSimulator\.synth_env_server; ./synth_env_server.exe Bedroom D. Set the Windows environment variable XR_RUNTIME_JSON to C:\Projects\MetaXRSimulator\meta_openxr_simulator.json.
Using Unity or Unreal Engine, run your game.
- For Unity: "Path to Unity\Unity.exe" -projectPath "ProjectPath" -testResults "test.log" -testPlatform PlayMode -logFile "log.log" -executeMethod GameManager.StartGame

Play the game in GameManager.StartGame, for example:

#if UNITY_EDITOR
using UnityEditor;
#endif
using UnityEngine;
using UnityEngine.SceneManagement;

public class GameManager : MonoBehaviour
{
    // Start the game automatically
    public static void StartGame()
    {
#if UNITY_EDITOR
        if (!EditorApplication.isPlaying)
        {
            EditorApplication.isPlaying = true;
        }
        else
        {
            EditorApplication.isPlaying = false;
        }
#else
        Debug.LogError("Cannot toggle play mode outside of the Unity Editor.");
#endif
    }
}

For Unreal: "Path to Unreal\UnrealEditor.exe" "Path to Test Project\test.uproject" -Game -ExecCmds="Automation RunTests Project.Functional Tests; quit" -Unattended -TestExit="Automation Test Queue Empty" -ReportExportPath="C:\temp\Reports" -Log -Log=RunTests.log
- Alternatively, you can compile your Unity/Unreal game into a game.exe file and run the executable, which will automatically execute the tests.

Repeat the above steps to run all your recordings.

Compare test results

Copy replay_expect.vrs and replay_new.vrs to a folder e.g. c:/test_result
Copy vrs_pixmatch.py and requirements.txt to a folder e.g. c:/test_result
Run below commands to compare the test results:

# Prepare requirements.txt for windows:
findstr /v "vrs==1.0.4" c:\test_result\requirements.txt > c:\test_result\tmp.txt
move c:\test_result\tmp.txt c:\test_result\requirements_windows.txt

# Download pyvrs
mkdir c:/tmp
cd c:/tmp
git clone https://github.com/facebookresearch/pyvrs.git
cd pyvrs
git submodule sync --recursive
git submodule update --init --recursive

# Build and Run Pyvrs via Pixi
# Install pixi (details can be found: https://pixi.sh/latest/#__tabbed_1_2)

iwr -useb https://pixi.sh/install.ps1 | iex

pixi run install_pyvrs

pixi task add run_test "python c:/test_result/vrs_pixmatch.py c:/test_result/replay_new.vrs c:/test_result/replay_target.vrs --threshold 0.2 --diffs_output_path c:/test_result"


pixi task add pip_install "pip install -r c:/test_result/requirements_windows.txt"
pixi run pip_install
pixi run run_test

More detailed information about the command can be found in the readme.md file under the scripts folder.

The above command will verify the test result and output the final result.

Update the test

If a game object has been changed from red to green, the test will fail. In this case, the recording is still valid, so you only need to run a replay locally to generate a new replay_expect.vrs file to update to the test machine (or CI). However, if the entire game interaction has changed and the recording is no longer valid, then you need to record a new test.

Test with more synthetic environments.

Synthetic environments are rooms for you to test your game, we have 3 built-in rooms and more are coming soon. They can be launched through the following commands:

cd C:\Projects\MetaXRSimulator\.synth_env_server; ./synth_env_server.exe Bedroom
cd C:\Projects\MetaXRSimulator\.synth_env_server; ./synth_env_server.exe Gameroom
cd C:\Projects\MetaXRSimulator\.synth_env_server; ./synth_env_server.exe Livingroom

Did you find this page helpful?