Health and Safety Recommendation: While building mixed reality experiences, we highly recommend evaluating your content to offer your users a comfortable and safe experience. Please refer to the Health and Safety and Design guidelines before designing and developing your app using Scene.

Samples

You can check out Phanto for an example of Scene Mesh being integrated into mixed reality gameplay.

Overview

The scene mesh is a low-fidelity, high-coverage artifact which describes the boundary between free and occupied space in a room. It is generated automatically during the space setup experience, and available for applications to query via Scene API.

Previously, Scene Capture was a manual process where a user outlined walls and objects in the room. Space setup will automatically capture structural room elements (walls, ceiling, floor), known as a Roombox, by creating an on-the-fly mesh reconstruction of the entire space and attempt to discover what part of this reconstruction is a wall, ceiling, or floor. This reconstruction yields a single mesh, known as the scene mesh, which can be accessed in Unreal after the space setup process has been completed.

Use cases

Fast Collisions: When used as a collider, the scene mesh allows virtual content to collide against real objects. Bouncing balls, projectiles, and other forms of fast collisions benefit from the high coverage of the room.

Obstacle Avoidance: When used for intersection checks or as part of a navmesh configuration, the scene mesh allows you to know where obstacles in the room are. This can inform content placement and AI navigation.

Prerequisites

In order to get this up and running, you will need to have the following:

A Quest 3 device

OS: v54

SDK: v56 Meta XR plugin

Setup a VR-ready Unreal project.

Capture a room using space setup.

Settings > Physical Space > Space Setup > Set Up

Request the USE_SCENE permission inside your application

Scene overview

Scene Model is a collection of objects called scene anchors, each of which have any number components on them that provide further information. Components hold information such as whether the anchor is a plane, whether it’s a volume, whether it has a mesh, whether it’s locatable, whether it’s a room, whether it holds other anchors, etc; the idea is that anchors are very generic objects, and an API user queries the components on the anchor to find what information they have.

For example, if you have a Scene Model of four walls, then you will have four scene anchors. Each anchor will be Locatable, have a Semantic Classification of WALL, and be a Plane which holds dimensions.

Spatial data permission

An app that wants to use Scene API needs to request Spatial Data permission during the app’s runtime. See the Unreal section of Spatial Data Permission and Requesting runtime permissions for more information.

Steps to create a scene mesh

In order to get the scene mesh in Unreal, you will need to do the following steps:

Create a pawn configured for VR and add it to the scene. (optional)

Add an OculusXRSceneActor to the scene and give it an OculusXRSceneGlobalMesh component.

Configure the OculusXRSceneGlobalMesh component.

Request permission for Scene.

Build and run your app - an Actor for the Scene Mesh will be spawned when the scene is loaded.

We will now look at each of these steps in greater detail.

1: VR ready pawn (optional)

After following a typical Unreal setup for Meta Quest development, you should start by adding a VR-ready pawn into the scene. This will add a complete camera setup to the scene, allowing you to move around the scene with the headset. You can also add passthrough to your scene by following these instructions.

2: OculusXRSceneActor

Add the OculusXRSceneActor into the scene. This class wraps all the functionality of Scene into a single class with easy-to-use SceneAnchorProperties that will be instantiated for those objects. The split is the following:

For elements that are strictly 2D (walls, floor and ceiling), objects from ScenePlaneSpawnedSceneAnchorProperties will be instantiated.,

For elements that have any 3D component (mesh or volume), objects from SceneVolumeSpawnedSceneAnchorProperties will be instantiated.

Scene mesh is managed by the OculusXRSceneGlobalMesh component, which needs to be a child of the OculusXRSceneActor. You can add it directly from the scene by clicking on the “+ Add” button in the Details panel of the OculusXRSceneActor and searching for “Global Mesh”.

You can add it directly from the scene by clicking on the “+ Add” button in the Details panel of the `OculusXRSceneActor` and searching for “Global Mesh".

3: Configure OculusXRSceneGlobalMesh component

In the OculusXRSceneGlobalMesh component, you can configure the Scene Mesh for collision by selecting the desired Collision Profile. You can also determine its visibility by choosing the appropriate material. It’s recommended to use a material with the Wireframe option enabled for displaying the Scene Mesh. To do this, navigate to Material > Advanced > Wireframe.

It's recommended to use a material with the **Wireframe** option enabled for displaying the Scene Mesh.

4: Request permission for Scene

In order to have the Scene Mesh working, you need to have the permission to use Scene.

There’s two ways to do so :

Enable Scene Support in your project settings: Meta XR/Mobile, MetaXR/Experimental.

Ask for the permission at runtime using the Request Android Permissions node. Asking for permission to enable scene support at runtime using the Request Android Permissions node.

Asking for permission to enable scene support at runtime using the Request Android Permissions node.

5: Build and play

Now that everything has been set up, compile the app and install it to your headset to see the app. There are various ways you can install the application, but all require your device to be connected to your computer with developer mode enabled.

Install it directly from Unreal: Click on Platforms then select your device. Install directly from Unreal by clicking on Platforms, then select your device.