The Meta Quest 3 and Meta Quest Pro use "Space Setup," a method where the headset's cameras and depth sensors create a 3D scan of the environment. This scan allows developers to craft virtual content that interacts realistically with physical objects and responds with accurate physics.
The Passthrough, Scene, and Anchor APIs are essential for enabling Space Setup and dynamic mixed reality experiences. With Passthrough, apps can utilize the device’s external cameras to capture physical environments in realtime, allowing people to see their surroundings in-headset. After obtaining a feed of a physical space, using the Scene API enables apps to identify and understand physical objects like doors, floors, and ceilings. By understanding physical objects, virtual content can interact and collide with physical objects as if they are in the same space to enhance user feelings of presence in-headset. Using Spatial Anchors, your app can ensure virtual objects maintain their position during and between sessions to deliver a more consistent and delightful user experience.