Explore a New Era of Mixed Reality with the Passthrough Camera API

Today we’re unlocking a new era in mixed reality development on Meta Horizon OS with the public release of the Passthrough Camera API in v76 (Unity | Unreal/Native | Android/Spatial SDK) with browser support coming soon. Apps leveraging the Passthrough Camera API can now be published to the Meta Horizon Store.

This game-changing capability provides access to the front-facing RGB cameras from Meta Quest 3 and 3S devices so developers can unlock new use cases leveraging object tracking, machine learning and computer vision, in addition to improving existing use cases through image or texture transfer support and enhancing app quality.

Based on the Android Camera2 API, the Passthrough Camera API is easily accessible to developers for nearly all supported build paths, including Unity, Unreal, Native engines, Meta Spatial SDK,Android panel apps, and web. Developers with apps containing augmented reality features based on the backward-facing camera should find it easy to enable on Quest devices.

Keep reading below to discover sample projects that can help you get started along with real examples of developers using the Passthrough Camera API to build for exciting new use cases.

Since introducing the experimental version of the Passthrough Camera API to select developers, we have been blown away by what they’ve built in a short amount of time. At GDC, a couple of these early teams showed off some eye-catching use cases, including:

Niantic Spatial shared how developers building with its SDK can leverage the Passthrough Camera API to build richer, more immersive experiences that understand and respond to the physical world in real time. By supporting live meshing, semantic segmentation, Visual Positioning Systems (VPS) and object detection via access to Quest’s front-facing cameras, Niantic Spatial helps developers build next-gen tools, games, and location-based content across consumer, enterprise and public sector use cases. Below, you can see how the API supports real time object recognition.

Niantic also highlighted the API’s value to visual positioning systemsthat aid navigation and transform physical spaces. In the wayfinding use case example below, a virtual character guides a user to a designated area within a populated, large-scale area where GPS may be ineffective. The experience not only maps out the physical space and navigates through it accurately, but modifies it with 3D textures that bring new life to a familiar journey.

Resolution Games showcased an early use case in which friends’ faces were transferred onto virtual ragdolls that can be picked up and tossed around. The Passthrough Camera API enables you to improve gameplay and create novel game experiences by incorporating textures and images from the physical world.

On social media, we’ve seen our community of developers show off demos demonstrating other ways in which the Passthrough Camera API enriches gameplay, including making game environments more realistic by adapting to the textures in users’ immediate vicinity, triggering context-aware gameplay via object recognition, making user experiences feel more custom and personal for each user and much more.

Passthrough Camera API also unlocks a new wave of dynamic and adaptable apps that were not possible before. Developers are applying these capabilities to a wide range of verticals, from education and design to industrial applications like safety training, construction and maintenance. Some examples include:

To help illustrate a real-world utility use case, CGS Immersive shared how it’s leveraging the Passthrough Camera API to analyze and troubleshoot servers. Using the API, CGS Immersive can better support its telecom partners by accelerating their provisioning and installation procedures with manager screenshots, compliance proofers and “taken path” job documentation.

In the example above, you can see how a technician is able to use Meta Quest 3S as a viewer to ensure that fiber cables are correctly installed into a server. The system analyzes the images gathered from the Quest 3S cameras to see if any fiber is in the wrong port and provides visual queues that help the technician check their work in real time.

Prior to the Passthrough Camera API, CGS Immersive’s partners needed to take and upload pictures separately on their mobile devices. Now, they can streamline their process by donning Meta Quest devices in Passthrough mode and either place or draw boxes on-screen to initiate necessary paperwork and workflow—and this works both in the cloud and locally if there’s no internet. This process leads to faster performance, better safety with robust AI support using LLMs like Llama 4 and ultimately happier tech installers who can get more done faster.

Developers can accelerate their process by leveraging sample projects, code and best practices that shorten the learning curve.

Gaining access to front-facing RBG cameras comes with a heightened responsibility to ensure that users’ spatial data is being managed and protected appropriately. The Passthrough Camera API is built first and foremost with privacy and security in mind, which is why developers must adhere to strict privacy requirements including app permissions and providing users with the ability to opt-in or out of allowing camera access.

When camera access is requested by your app, users will see a permission opt-in prompt. Upon opting-in, recording indicators will become visible so users know that the feature is turned on. By providing user controls and being transparent, we can help alleviate potential concerns, build trust and maintain data integrity so your audience can remain present and engaged with your experience.

The Passthrough Camera API is a long awaited feature that unlocks new possibilities to enhance apps and push the limits of mixed reality by supporting machine learning and computer vision. To get started, visit the documentation (Unity | Unreal/Native | Android/Spatial SDK, and stay tuned for browser/WebXR support. Don’t forget to share your projects on social with the #PCAonQuest hashtag so we can follow your journey and see how you’re bringing your apps to life with the Passthrough Camera API.

For the latest developer news and feature launches, follow us on X and Facebook and subscribe to our monthly newsletter in your Developer Dashboard settings.