Scene Overview

Health and Safety Recommendation: While building mixed reality experiences, we highly recommend evaluating your content to offer your users a comfortable and safe experience. Please refer to the Health and Safety and Design guidelines before designing and developing your app.

Scene empowers you to quickly build complex and scene-aware experiences with rich interactions in the user’s physical environment. Combined with Passthrough and Spatial Anchors, Scene capabilities enable you to build Mixed Reality experiences and create new possibilities for social connections, entertainment, productivity, and more.

There are two important concepts in Scene:

Mixed Reality Utility Kit provides a rich set of utilities and tools on top of the Mixed Reality APIs, and is the preferred way of interacting with Scene.

Space Setup (formerly known as Scene Capture) is a system flow that lets users walk around and capture their scene to generate a Scene Model. Users have complete control over the capture experience and can decide what they want to share about their environment. An app can query the system to check whether a Scene Model of the user’s space exists or invoke Space Setup if needed.

Scene Model is a single, comprehensive, up-to-date representation of the real physical world that is easy to index and query. Scene Model provides a geometric and semantic representation of the user’s space so you can build room-scale mixed reality experiences. For example, attach a virtual screen to the user’s wall or have a virtual character navigate on the floor with realistic occlusion. In addition, you can bring physical world objects into virtual reality. For example, users can see their real desk or couch in the virtual world to play or work more comfortably.

The fundamental elements of a Scene Model are Scene Entities where each entity comes attached with geometric components and semantic labels. For example, the system organizes a user’s living room around individual anchors with semantic labels. These anchors include the floor, ceiling, walls, desk, and couch. Each anchor is also associated with a simple geometric representation, 2D boundary, or 3D bounding box.

Scene Model is managed and persisted by Meta Horizon OS. All apps can access the Scene Model. Scene Model makes it easy to understand the environment around the user and blend virtual content into that environment.

You can use Scene Models in Unreal to build mixed reality apps with physics, occlusion, and navigation effects. You can query specific elements in the Scene Model or use the entire Scene Model.

Scene Mesh is a triangle mesh representation of the room boundary generated from the Scene Model. It enables fast collisions, obstacle avoidance, and realistic occlusion by providing a detailed geometric surface of the user’s environment. Through Mixed Reality Utility Kit, you can access and use Scene Mesh in your mixed reality experiences. For more details, see the Scene Mesh page.

Space Setup allows users to scan and maintain multiple rooms. Meta Horizon OS can store multiple rooms and locate some or all of them based on the user’s current location. Multiple rooms are supported by default across all platforms.

The recommended way to access Scene API is to use Mixed Reality Utility Kit. Refer to the Getting Started section in the Mixed Reality Utility Kit documentation for more details on how to get your project setup. If you already have a project using OculusXRSceneActor, see Get Started with Scene Actor. You can also find deprecated Scene Actor documentation in the left sidebar under Manage Scenes > Deprecated.

An app that wants to use Scene and Depth API needs to request Spatial Data permission during the app’s runtime. See the Spatial Data Permission page and Requesting runtime permissions⁠ for more information.

Now that you have a broad overview of Scene, learn more by digging into any of the following areas:

Design guidelines are Meta’s human interface standards and design frameworks that help you create safe, user-oriented, and retainable immersive and passthrough user experiences.