Develop

Build Your Own Synthetic Environment Server

Updated: Apr 22, 2026

Unity-based tool

The Synthetic Environment Builder is a Unity Editor tool. The environments it creates are compatible with Unreal Engine apps through the Meta XR Simulator. This page explains the cross-engine workflow and links to the full Unity-based build instructions.

Overview

The Synthetic Environment Builder lets you create custom synthetic environments for the Meta XR Simulator. It is a Unity Editor tool distributed as a UPM (Unity Package Manager) package. You use it to build a Synthetic Environment Server from a Unity project containing 3D scene content.

The environments created with the Builder work with apps built in any engine, including Unreal Engine, Unity, and Native. This is possible because the Meta XR Simulator operates as an engine-agnostic OpenXR runtime⁠. Your Unreal Engine app connects to the XR Simulator, which in turn connects to the Synthetic Environment Server to provide Passthrough, Scene, and Depth data.

For the full step-by-step build workflow, see Build your own synthetic environment server (Unity).

How it works with Unreal Engine

The Meta XR Simulator uses a client-server architecture:

Synthetic Environment Server: A standalone application built using the Unity-based Synthetic Environment Builder. It provides the 3D environment data (Passthrough, Scene, and Depth).

Client application: Your Unreal Engine app, which connects to the Meta XR Simulator. The XR Simulator acts as an OpenXR runtime and forwards environment data from the server to your app.

You do not need Unity installed to run a pre-built Synthetic Environment Server. You only need Unity to create or modify environments using the Builder.

Environment orientation rules

When building a custom environment with the Synthetic Environment Builder, the following orientation rules apply to scene entities. These rules are engine-agnostic and affect how Scene data appears in your Unreal Engine app:

For ceilings, floors, and walls, +y must point into the room.

For walls, doors, and windows, +x must point right, +z must point up, and +y must point into the room.

For other panels, +y is the up direction.

Additional constraints:

Each scene must have exactly one ceiling and one floor.

Copy-pasting a bounded 2D entity plane is not supported.

Alternatives for Unreal Engine developers

If you do not need a full Synthetic Environment Server, the Meta XR Simulator provides other options for testing mixed reality features:

Built-in rooms: The Meta XR Simulator ships with pre-built synthetic environments (game room, living room, bedroom, and more). These provide Passthrough, Scene, and Depth data without additional setup. See Meta XR Simulator Passthrough Scenes.

JSON-based rooms: Lightweight, text-based room definitions that provide Scene data only. These are useful for automated testing and do not require a running server. See Meta XR Simulator JSON Format Rooms.