Presence Platform Voice SDK and Passthrough Now Available

At Connect 2021, we introduced Presence Platform—a broad range of machine perception and AI capabilities that allow you to build more realistic mixed reality, interaction, and voice experiences that seamlessly blend virtual content in a user’s physical world.

Today, we’re excited to announce the experimental release of Voice SDK, part of the Presence Platform, on Quest devices. Also, you can now start building and shipping mixed reality experiences with the general availability of Passthrough.

Voice SDK is a set of natural language capabilities that let you create hands-free navigation and new voice-driven gameplay. With Voice SDK, you can create Voice Navigation & Search to help people get to where they want to go more quickly, or enable Voice FAQ to allow users to ask for help or a reminder. We’re also enabling new Voice-Driven Gameplay—like winning a battle with a voice-activated magic spell or talking with a character or avatar. Voice SDK is powered by Meta’s Wit.ai⁠ natural language platform.

For this experimental release, we're supporting English voice recognition, as well as offering an early preview of voice recognition in other languages that Oculus supports. Voice SDK Experimental has a rich set of 50+ common built-in intents, entities and traits and supports full customization, for voice use cases in VR.

Voice SDK is available for Unity developers in this v34 SDK release, with support for other development platforms coming in the future. This experimental API allows you to start building and testing Voice-enabled experiences on Quest 2. The production version of Voice SDK is targeted for later this year, and at that point you’ll be able to ship these experiences to your users.

To learn more or if you missed the Voice Interactions in VR session at Connect 2021⁠, you can watch it here:

Download Voice SDK Experimental in the v34 SDK release and Voice SDK developer documentation.

Need inspiration? Check out the Voice SDK in action with FitXR:

With this SDK release, we’re also announcing the general availability of Passthrough—a way to build and ship apps that seamlessly blend the real and virtual worlds and open up new possibilities for mixed reality experiences. With this v34 SDK release, Passthrough will allow you to create and ship new experiences for Productivity and Gaming.

If you missed the Passthrough session at Connect 2021, you can watch it here:

You can now download Passthrough in our v34 SDK release and access Passthrough API developer documentation.

Check out Passthrough in action with MagicKeys:

We’re excited to see the mixed reality experiences you build with Presence Platform for Quest devices. With these new capabilities, and even more coming soon, we can begin to explore what the metaverse might look like, and we’re committed to helping you create the connected, interoperable worlds that lie ahead.

If you have any questions, please let us know in the Developer Forums⁠.