Design

Blend virtual objects with the physical environment

Updated: Oct 8, 2025

What is passthrough?

Using passthrough, users can see their physical surroundings through a video feed from the Meta Quest’s cameras. These experiences blend digital objects with physical environments.

Passthrough is not a simple video feed displayed to the user. To deliver realistic, comfortable, and natural viewing experiences, it is essential that passthrough provides realistic depth and proper distortion. Simple monoscopic video (i.e., seen from a single viewing angle) fails to convey a sense of depth, and can be uncomfortable or unsafe for users.

An example of blending a digital character with reality using passthrough

Figure 1: Virtual content in the physical environment

Why is passthrough important?

Passthrough is a crucial element of mixed reality because it provides a video feed of the physical environment that can be combined with digital objects. Being able to use the physical environment around the user opens up exciting new possibilities for designing content and interactions. For example, a digital recipe can be overlaid on a physical surface in a kitchen, or a movie can be watched while a user is interacting with their family.

How can passthrough be used?

Passthrough as a background

When the passthrough feed is used as an entire background, users can see their 360 degree surroundings in the physical environment, just like wearing see-through glasses.

Video 1: An example of passthrough background with Meta Quest user interface

Increase physical environment awareness

Passthrough helps build apps that improve physical environment awareness while enjoying virtual experiences. For example, the productivity application below shows virtual office tools within the user’s actual, physical workspace.

A productivity example, showing how passthrough can help create large, digital workspaces

Passthrough windows and portals

The selective passthrough capability uses a passthrough feed on a specific portion of a fully immersive experience. For example, magical windows can be overlaid onto the physical world while in fully immersive experiences. This can help show only partial windows into the user’s physical environment, and also helps frame transitions between physical and fully immersive experiences. The following image shows clear separation along natural borders, or full screen blends where virtual and physical spaces align - revealing delightful experiences in a comfortable manner. Another way of only making passthrough visible within the specified surface geometries, is to use surface-projected passthrough which allows apps to specify the geometry onto which the passthrough images are projected.

Passthrough with scene understanding

When passthrough is used with scene understanding, virtual objects can be shown interacting with physical surfaces. For example, a character can walk on any surface, including walls and tables. By combining passthrough and scene understanding, the physical environment can be used as a design canvas.

Video 2: This example demonstrates how virtual objects can interact with various types of physical surfaces

Switching between passthrough and fully immersive experience

If an app requires frequent switching between the passthrough and fully immersive experiences, consider providing an easily accessible toggle-switch UI, such as a main menu or manual menu. The abrupt transition between virtual and physical environments can be an overwhelming and uncomfortable experience, especially when the two environments look vastly different. Because of this, it’s a good idea to apply smooth blending when transitioning between the two.

Instead of completely turning the passthrough feed on-and-off, you can also control the opacity of the passthrough feed. This means, you can blend the physical and virtual experiences smoothly. As described in the section above, be mindful that overlapping two distinct and incongruent realities can be confusing for users. Controlling the opacity of passthrough can be useful when the virtual environment aligns 1:1 with the user’s physical environment.

If passthrough is not a critical part of your app, and does not require frequent switching, the toggle option can be provided in the settings menu.

Video 3: This example illustrates a smooth transition between the virtual and physical environment in Meta Quest settings

Passthrough (contextual) splash screen

For apps that launch directly into a passthrough view, the user experience is typically much better if the loading screen shows passthrough in the background. The Meta Horizon OS can detect if the user is in passthrough mode in their home environment before the user loads the app. If so, the splash screen that appears when loading the app can be displayed in passthrough as well, which is called contextual passthrough.

Best practices

Clearly signal reality transitions

When presenting users with multiple realities, a good rule of thumb is to separate them and clearly mark the transition. The exception to the above rule is when a 1:1 match exists between virtual and physical spaces. In these cases, the effect can be magical. Generally speaking, overlapping two distinct realities (the physical and virtual worlds) can be confusing for users, and is discouraged. The The World Beyond sample app demonstrates a good example of delightful transition between the passthrough physical world and virtual world with animation and particle effects.

Blending the physical world with the virtual world

Be mindful that overlapping two distinct and incongruent realities (the physical and virtual worlds) can be confusing for users, and is discouraged. The following examples illustrate how the failure to separate and clearly mark the transition from virtual to physical spaces leads to a jarring, disorienting experience. This can cause users to feel confused and unsafe. In both images, a virtual grid overlays the physical world, making it difficult for a user to understand which reality to follow.

An example of passthrough creating confusion between digital and physical realities

The following image show clear separation along natural borders, or full screen blends where virtual and physical spaces align - revealing delightful experiences in a comfortable manner.

An example of passthrough being well defined with clear separation along natural, physical borders

The World Beyond sample app demonstrates a good example of delightful transition between the passthrough physical world and virtual world with animation and particle effects.

Video 4: Example of switching between the physical world and virtual world only on specific surface area of the environment

Disable the boundary when in passthrough

The Boundary API allows us to turn off the boundaries contextually. This means it turns on the boundaries when in fully immersive experiences and disabled the boundaries when the user sees their environment in passthrough.

Passthrough and colors

Depending on the Meta Quest type, passthrough may be displayed in color or grayscale. The images below show examples of Meta Quest 2’s grayscale pass-through (left) and Meta Quest Pro’s color passthrough (right).

Meta Quest 2's grayscale feed is shown on the left, and Meta Quest Pro's color passthrough is shown on the right

Passthrough can be shown as-is or stylized to achieve an artistic look. If using Meta Quest 2 with grayscale passthrough, certain considerations need to be kept in mind:

When using passthrough on Meta Quest 2, consider using a visual language where grayscale represents the physical world and color represents the virtual world. This way, colors will drive attention toward gameplay or storytelling elements.

Stylization effects can be animated, providing means to highlight particular moments to the user, such as transitioning from the physical world to a virtual one, or reaching an achievement.

In the following image, a portal is about to open in the middle of the room. Color is used to draw the users attention.

Color is used to draw the user's attention to where a portal is opening

On Meta Quest Pro, you can leverage a color passthrough video feed of the environment. It becomes easier to blend physical and virtual worlds together with this added level of detail.

Lighting and dimming

Hardware capabilities allow you to control the blending between passthrough and your app. By playing with the transparency of a black background, applications can create the illusion that the environment is actually being lit and dimmed.

Lighting passthrough is still an open area for exploration, with no single way to create this type of illusion. You can experiment by blending the gradient mask, grayscale re-lighting, tinting the passthrough images, adjusting the camera’s background opacity, or stylizing passthrough edges to achieve the desired effect.

The following examples demonstrate lighting with passthrough. In the left-hand image, passthrough is dimmed everywhere, except for a cone of light attached to a flashlight that the user holds in their hand. In the right-hand image, passthrough is dimmed around a board game on a table, drawing the user’s attention to the game.

An example of dimming passthrough to draw the user's attention to a board game

The World Beyond shows a dimmed environment, and encourages the user to focus on the portal where multi toy shows up. When the user grabs the multi toy, passthrough gradually adjusts the lighting with animation, creating a comfortable, intuitive experience.

Video 5: An example of how The World Beyond utilizes changes in lighting to focus the user and create comfortable transitions

Create smooth transitions between passthrough and fully immersive experience

Design seamless transitions between experiences by using gentle, predictable animations that help users visually track changes. Allow users to control their level of immersion by providing clear entry and exit points, such as prominent buttons or controls. Ensure that exiting an immersive experience is intuitive and doesn’t result in unexpected consequences, like quitting the app or game without saving progress. Consider allowing the user to customize the immersion range for an experience to enhance the user experience.

Did you find this page helpful?