Scene Overview

Health and Safety Recommendation: While building mixed reality experiences, we highly recommend evaluating your content to offer your users a comfortable and safe experience. Please refer to the Health and Safety and Design guidelines before designing and developing your app.

Scene empowers you to quickly build complex and scene-aware experiences with rich interactions in the user’s physical environment. Combined with Passthrough and Spatial Anchors, Scene capabilities enable you to build Mixed Reality experiences and create new possibilities for social connections, entertainment, productivity, and more.

There are two important concepts in Scene:

Mixed Reality Utility Kit provides a rich set of utilities and tools on top of the Mixed Reality APIs, and is the preferred way of interacting with Scene.

Space Setup (formerly known as Scene Capture) is a system flow that lets users walk around and capture their scene to generate a Scene Model. Users have complete control over the capture experience and can decide what they want to share about their environment. An app can query the system to check whether a Scene Model of the user’s space exists or invoke Space Setup if needed.

Scene Model is a single, comprehensive, up-to-date representation of the real physical world that is easy to index and query. Scene Model provides a geometric and semantic representation of the user’s space so you can build room-scale mixed reality experiences. For example, attach a virtual screen to the user’s wall or have a virtual character navigate on the floor with realistic occlusion. In addition, you can bring physical world objects into virtual reality. For example, users can see their real desk or couch in the virtual world to play or work more comfortably.

The fundamental elements of a Scene Model are Scene Entities where each entity comes attached with geometric components and semantic labels. For example, the system organizes a user’s living room around individual anchors with semantic labels. These anchors include the floor, ceiling, walls, desk, and couch. Each anchor is also associated with a simple geometric representation, 2D boundary, or 3D bounding box.

Scene Model is managed and persisted by the Meta Horizon operating system. All apps can access the Scene Model. Scene Model makes it easy to understand the environment around the user and blend virtual content into that environment.

You can use Scene Models in Unreal to build mixed reality apps with physics, occlusion, and navigation effects. You can query specific elements in the Scene Model or use the entire Scene Model.

Starting from v62, Space Setup allows the user to scan and maintain multiple rooms (spaces) instead of a single room. The user can scan a new room without erasing a previous room. The OS maintains up to 15 rooms, and might be able to locate some or all of the rooms depending on the current user’s location.

Our SDKs for all platforms (OpenXR, Unity, Unreal) have been designed to support multiple rooms. However, Space Setup before v62 provided only a single room, so your existing applications might not behave as expected when more than one room is available. Existing applications will not have multiple rooms returned, starting from v65, new SDK versions will support multiple rooms being returned by default.

As of v65, the recommended way to access Scene API is to use Mixed Reality Utility Kit. Refer to the Getting Started section in the MR Utility Kit documentation for more details on how to get your project setup. If you already have a project using OculusXRSceneActor then you can find the documentation for it in the Additional Resources > Deprecated section.

An app that wants to use Scene and Depth API needs to request Spatial Data permission during the app’s runtime. See the Spatial Data Permission page and Requesting runtime permissions for more information.

Now that you have a broad overview of Scene, learn more by digging into any of the following areas: