An immersive experience is an umbrella term that covers the spectrum between full immersion and seeing some virtual elements superimposed on your physical surroundings. It includes both fully immersive and mixed reality experiences.

In mixed reality, the digital and physical worlds are blended, allowing users to interact with digital objects while seeing their physical environment.

This means mixed reality can open the door to a variety of unique experiences such as games and apps that respond to and incorporate your physical space. Imagine seeing shooting stars and constellations on your ceiling, or a game character bursting through a wall of your room.

Mixed reality gives people the freedom to see and interact with people and objects in their physical space while enjoying virtual content. It also makes it easier to move around without needing a boundary of clear space. Thanks to a high degree of usability, mixed reality unlocks a new class of experiences that can transport you from the familiar to the fantastical in an instant.

In this type of virtual experience, a user is fully surrounded by the virtual world. Since it blocks out physical surroundings and allows the user to engage deeply with the content, it is best suited for activities that require full immersion, such as gaming, simulations, and 360-degree videos.

Since users cannot see their surrounding environment while using the system, boundary visualization is provided to prevent collisions with objects or the environment in physical space. The boundary ensures a safe and immersive experience.

Unlike fully immersive experiences, in mixed reality a user is not entirely immersed and can see their physical environment, or the virtual content around them is mapped to their physical surroundings. This type of experience is ideal for providing contextual information and creating interactive experiences within the physical environment.

The device is aware of physical surroundings and surfaces, allowing designers to use physical space as a canvas. For example, a virtual object can be placed on a table or hung on a wall, just like a physical object, allowing users to view and interact with it from multiple directions.

The video below shows some examples of how mixed reality can be used. For instance, a game character can walk around in a physical environment, a virtual game board can be placed on a table, information can be overlayed on top of physical objects, and more.

Use the following as building blocks to create mixed reality experiences.

Passthrough blends digital objects with the user’s physical environment. When passthrough is enabled, the user can see their actual surroundings through the device camera’s video feed. This is not just a simple video feed, it is a realistic and natural viewing experience that provides realistic depth and proper distortion.

Users can adjust the visibility of their surroundings by switching from one mode to another by the press of a button or gradually by using a slider while in an experience. Users can view 3D content blended with their surroundings, open a portal to view another place, or enter a different world.

Create mixed reality experiences for users by using passthrough as an entire background. Use passthrough on specific areas to build unique experiences like portals and magical windows.

Read our detailed design guidelines for passthrough.

The scene understanding capability allows digital objects to interact with physical surfaces in a user’s physical environment. Design these types of interactions using the associated Scene API. For example, use the Scene Model to make a virtual ball bounce off of a physical table in a room or have a virtual robot scale physical walls.

This capability allows the representation of different surfaces as 2D planes or 3D volumes. In addition to this, it also provides semantic labels for these surfaces such as the floor, ceiling, walls, desk, couch etc.

Use this semantic information about the type of surface to create exciting mixed reality experiences. For example, open up a ceiling and turn it into a dark sky with the Milky Way and stars, or have a character walk on the floor and then sit on a sofa.

Additionally, with Meta Quest’s depth sensors, achieve environment occlusions and dynamic environment collisions without the need to scan the scene beforehand. (Meta Quest 3 only)

Check out our detailed design guidelines for Scene understanding.

Spatial anchors are a concept that enables anchoring of virtual objects or content to a specific location or object in the physical environment. Spatial anchors can be thought of as an “anchor” that ties a virtual object to a specific point in space, allowing it to remain in place even as the user moves around it. This is achieved through various technologies such as computer vision, machine learning, and sensor fusion.

With spatial anchors, place any type of digital object on a specific, physical object or location (such as the wall in a room) and have the object appear on that wall every time a user returns to the same mixed reality space. For example, designers can use spatial anchors to anchor several 3D schematics in place in Gravity Sketch, or a group of friends can use it to anchor the game world in Demeo to their table so they can return to the same setup and continue their game anytime.

Use spatial anchors in combination with semantic surfaces (such as the walls, a table, the floor, etc.) to create rich experiences and automatic placement at room-scale. The user’s room becomes a spatial rendering canvas for both gameplay and productivity. Anchor a virtual door to a physical wall which the user can then open to reveal a fully immersive virtual world, complete with its own characters that can then enter the user’s physical space.

Shared spatial anchors allow anchors created by one person to be shared with others in the same physical space. Build local multiplayer experiences by creating a shared world-locked frame of reference for multiple users. For example, two or more people can sit at the same physical table and play a virtual board game on top of it.

See Spatial anchors for more detailed design guidelines.

Mixed reality revolutionizes the way users interact with virtual objects, transcending the limitations of 2D screens. Within a 3D space, whether physical or virtual, users can engage with virtual objects in a more immersive and interactive way. The Meta Quest platform accommodates a range of input modalities, each with unique capabilities and characteristics, empowering users to select their preferred input type based on content type, context, and task requirements.

This opens up thrilling opportunities for designers to craft innovative, immersive, and engaging experiences that seamlessly merge physical and virtual worlds. The Presence Platform further amplifies interactions with a diverse array of tracking capabilities, including hand, controller, eye, face and even body tracking, unlocking endless possibilities for creative expression and user engagement.

See Key considerations page for the best practices on spatial interactions. Check out pages under Core interactions such as Input modalities and Input primitives for more detailed design guidelines around input.

Ensuring visual comfort is crucial since users rely entirely on the headset’s cameras to see their physical and virtual environment. For mixed reality experiences it is important to dynamically adjust content based on the user’s head position and depending if the user is standing, sitting down, laying down, or in a moving vehicle. We want to allow the user to comfortably interact with the core experience without the need to physically move or manually adjust the content too much, for a more relaxed experience.

See Comfort for more detailed design guidelines.

Audio is crucial for creating a persuasive immersive experience and the ability to localize audio sources in three-dimensional space is a fundamental part of how we experience sound.

See Audio for more detailed design guidelines.

Meta Presence Platform offers several ways to make an experience more accessible. For one, there are various ways for developers to design their input to fit it to their user’s needs. Also, we offer a Voice SDK, which enables developers to easily set up voice support for navigating the experience in a more comfortable way.

See Accessibility for more detailed design guidelines.

Check out our design guidelines for other important elements of mixed reality. Learn what you can do with them, and how to design great mixed reality experiences:

For technical information, start from these development guidelines on Presence Platform’s mixed reality capabilities: