Introducing Mixed Reality Motifs: Passthrough Transitioning

With the launch of Meta Quest 3, the developer community is beginning to show the world why Mixed Reality can be special. We believe there has been more innovation in the past year than in the previous 10 years, and through this innovation many developers have already been able to create entirely new mechanics that accurately showcase why MR is the best way to engage Quest users. To help you accelerate development, we’ve been working to broadcast these widely embraced MR mechanics so you don’t have to start from a blank slate every time you want to build a magical MR experience.

We’ve coined these mechanics with the term ‘MR motifs,’ which serve as blueprints for recurring features that we expect other developers to gravitate towards as they start building new MR experiences. Motifs are not full apps, but rather recurring aspects that may require a specific collection of technical features and APIs to be achieved.

The first motif we are exploring in this blog post is from developers transitioning from fully immersive experiences to mixed reality experiences using Passthrough, or vice-versa. Below, we’ll address what Passthrough is, where and how it can be used, and how other developers who have shipped popular apps on the Meta Quest Store are leveraging the magic of Passthrough today. At the end of this blog post, you will be able to adjust the visibility of your surroundings by manipulating a slider, which regulates the level of Passthrough, or directly switch from one mode to another by the press of a button.

You can find this project with all the source code on GitHub. Also, don’t forget to check out our video tutorial on YouTube showing how to use this sample step-by-step!

Passthrough technology utilizes the device's external cameras to deliver real-time video, enabling users to engage with virtual elements while remaining aware of their real environment. It also means users will feel free and safe to move around within their space. Moreover, Passthrough goes beyond merely serving as a background element and allows creators to craft immersive experiences incorporating the user's room and furniture into the gameplay itself. We will explain these types of concepts in dedicated Motifs in the future.

Enabling passthrough is easy but there are some pitfalls and two main scenarios where a separate logic is needed for using passthrough in a Mixed Reality experience; contextual Passthrough and conditional Passthrough.

Contextual Passthrough refers to a scenario in which Passthrough should be enabled based on the system’s recommendation. In other words, if someone is in Passthrough mode in their home environment, our operating system can detect this and display the system splash screen as well as the Unity scene in Passthrough as well. In VR mode, the system splash screen will be shown with a black background.

A splash screen in Unity apps on Meta Quest is the initial screen that displays when the app is launched, often showcasing the app's logo or brand, before transitioning to the main content or interface. By default, splash screens are shown with a black background and display a Unity logo and/or an app-logo. For MR apps that launch directly into a Passthrough view, the user experience is typically better if loading screens show Passthrough in the background. The Meta OS already allows you to do this out of the box by going to the System Splash Screen Background option on the OVR Manager component in Unity and changing it to Passthrough (Contextual).

It’s important to make sure that “Show Splash Screen” is unchecked and that no texture is selected for the Virtual Reality Splash Screen in the Unity Project Settings. Keep in mind that currently this built-in splash screen can only be turned off with a Unity Pro license. Lastly, you will need to update the Android Manifest file in your project. To accomplish this, you will need to go to “Meta,” then “Tools” and update the Android Manifest file.

Depending on the system’s recommendation, you may want to deploy either a VR or Passthrough background. This recommendation is available using:

You might want to save this preference in a boolean variable, which later makes it easier to enable or disable the passthrough layer accordingly. Combining this approach with the contextual Passthrough system splash screen allows for an uninterrupted user experience.

Popular apps that use this technique typically leverage Passthrough as the mode of their core experience.

Popular apps that use this technique usually deploy a Passthrough background during their core experience. One example of this is Cubism, a puzzle game in which a user in Passthrough mode within their Meta Quest home environment can load the game with a contextual Passthrough splash screen and continue to play the actual experience in Passthrough without interruption. The video below shows this prime example of how to use the Passthrough system splash screen.

Another way to leverage Passthrough is using it to switch between MR and VR game modes, or to conditionally switch to Passthrough (e.g. when opening the menu, double tapping on the device, changing scenes, or to conditionally fade in and out certain parts of the experience, such as a keyboard, screen, wall, or ceiling). All these use cases require a way to switch between fully immersive VR and Passthrough conditionally on-command. Showing the whole environment or parts of it at selected times can invoke a sense of comfort and immersion for the player, leading to greater retention.

The most straightforward way of switching between modes is to simply enable or disable the Passthrough layer and change the clear flags and background color of the main camera. You will likely also want to implement a toggle method and introduce a boolean variable, such as “passthroughToggled,” to accurately toggle between modes on demand during runtime.

While this solution provides an easy method of alternating modes, if your app experience switches between Passthrough MR mode and immersive VR mode several times, you might want to find a more elegant way of transitioning.

It’s important to keep in mind that enabling Passthrough is asynchronous. System resources like cameras can take a few hundred milliseconds to activate, during which time Passthrough is not yet rendered by the system. This can lead to a black flicker if your app expects passthrough to be visible immediately upon enabling. You can avoid this hiccup by using the “PassthroughLayerResumed” event, which is emitted once the layer is fully initialized and Passthrough is visible. The goal is to smoothly transition between the two modes, rather than immediately going from Passthrough to VR.

A common way to fade between scenes or modes in XR apps is to use a fade screen, which usually consists of a quad that sits in front of the main camera. Typically, developers would then smoothly fade the material of the quad in and out, and you can use a similar approach when fading between Passthrough and VR. We have shipped a shader called “SelectivePassthrough,” which is part of the Meta Core SDK for Unity that enables you to selectively reveal Passthrough. You can also see a use case with this shader when using the Passthrough Window Building Block.

The shader allows you to turn the pixels on your screen transparent and reveal Passthrough. This is what you will use for your Passthrough fader shader. To achieve an even smoother fade, you will want to disable the premultiplied alpha mode on our OVR Manager with OVRManager.eyeFovPremultipliedAlphaModeEnabled = false; in the Awake() method.

Disabling eyeFovPremultipliedAlphaModeEnabled switches from premultiplied alpha blending to standard alpha blending. Premultiplied alpha blending can sometimes cause visual artifacts, such as flickering or halo effects, especially when dealing with semi-transparent pixels or certain lighting conditions. By disabling it, you will use standard alpha blending, which may handle these situations better and result in smoother visuals.

Taking all the learnings above, you can put everything together to achieve a smooth transition for your audience. We’ve described how to fade the material in and out by adjusting the _InvertedAlpha property; however, if you would like to achieve a gradient fade (e.g. from the bottom to the top, like in the Meta Quest home environment), you will need a new property in your shader called _FadeDirection. In the fragment shader (frag) you will then need to introduce a new variable to adjust the alpha value of the color.

Next, you will want to create a material from your shader, create a sphere on the main camera, and attach the material to this sphere. The fader logic should be attached to our sphere, so it can fetch a reference to the meshrenderer. The script should include a FadeDirection enum that defines the direction of the fade effect. The main part of the logic could be a toggle method that toggles the Passthrough layer and starts the fade effect. If the target alpha is 1, it starts fading out; if the target alpha is zero, it enables the Passthrough layer. You will want to bind this method to either a physical button or a UI panel.

You can either disable and enable your sphere everytime the fade is completed and switch the camera’s clear flags and background color. Alternatively, if your game requires it, you could keep the sphere enabled at all times, which prevents the need to switch the camera clear flags and background color every time you switch between modes. This has an additional benefit of being able to set a fixed distance of how far back you will see your virtual content in Passthrough mode as opposed to everything being visible when the quad’s renderer is disabled. Check out our YouTube tutorial for a more detailed explanation of the code!

One popular app that uses conditional Passthrough mode in a recommended fashion is Wooorld, which delivers a compelling user experience in part by switching from a Passthrough MR mode to a fully immersive VR mode. By enabling users to see the map models in Passthrough and then jumping into a location in a fully immersive mode, users can still be aware of their surroundings and enjoy a greater sense of delight versus being in full VR at all times.

While the boundary is an important safety system required for fully immersive apps, it can unnecessarily interrupt MR experiences. The Boundary API provides a mechanism to disable the boundary via Contextual Boundaryless, which allows your app to operate without a boundary for some or all of an experience while being in Passthrough. The Boundary API can be accessed through the OVRManager component in Unity.

In order to enable support for the Boundary API, it is necessary to choose either “Supported” or “Required” in the Quest Features of the OVRManager. Passthrough Support cannot be selected as “None.” Additionally, you will need to check the box called “Should Boundary Visibility Be Suppressed” under “Insight Passthrough & Guardian Boundary”.

To apply these changes to your APK, you will need to go to “Meta” then “Tools,” and then update the android manifest file. It is important to mention that you will solely disable the boundaries in Passthrough mode, and only when running the app on the device on your APK, and not in a fully immersive mode for safety reasons. Optionally, you can completely turn off the boundary. Boundaryless apps disable the boundary for the entirety of the experience and must not have any immersive experiences. It is critical to ensure that your app can safely disable the boundary for the entirety of the app experience.

Deeper feelings of presence have a direct correlation to user engagement. By deploying smooth and optimally-timed transitions between VR mode and Passthrough using motifs, you can elevate the unique aspects of your app and help people feel more engaged with their surroundings. You can find the source code to this guide here. For more best practices and troubleshooting, check out the official Passthrough documentation!