Immersive, Realistic Audio in VR: Introducing Acoustic Ray Tracing in Audio SDK

Immersion is a defining aspect of VR experiences, encompassing a combination of several factors that deliver greater feelings of presence and heightened sensation when compared to traditional 2D screens. While stunning visuals are often the first thing people notice, the importance of high-quality audio cannot be overstated. Accurate audio simulation has the power to produce 360-degree soundscapes that expand the virtual environment beyond the player's direct line of sight, transforming immersive environments into truly believable experiences.

To help you achieve accurate audio simulation, we’ve launched new Acoustic Ray Tracing (Unity | Unreal) capabilities as part of the Audio SDK. These capabilities give you the tools to increase realism, deepen engagement, anchor players in virtual worlds, and bridge the gap between remote multiplayer participants by giving the impression that they occupy the same physical space.

Typically, acoustics are implemented by developers with bespoke ad-hoc systems that use game logic to drive audio system parameters. They can often achieve basic occlusion and obstruction and apply different reverb presets for different areas, but this manual approach tends to be labor intensive, complicated and brittle.

While some solutions exist on the market to achieve highly accurate levels of acoustics simulation, these are intended for PC and console development, and are often too resource-intensive to be suitable for mobile VR hardware.

Now you can leverage the new Acoustic Ray Tracing features in Audio SDK to easily create realistic acoustics for virtual environments in Unity and Unreal. Simply load the virtual environment in the game editor, tag the relevant geometry, and follow a set of simple steps to automatically generate an accurate acoustic model.

We’ve worked to streamline this process to empower developers and designers with minimal experience in sound design to create authentic acoustics. Our system is powerful enough to handle large, complex environments with ease, performant enough to target Meta Quest mobile devices, and simple enough that artists can use it without requiring engineering expertise.

Dive in below to learn more about these exciting new features, how to implement them, and example use cases.

Our new Acoustic Ray Tracing features are designed to simulate high-quality audio acoustics with incredible accuracy. By pairing these capabilities with our existing spatial audio functionality, you can create environments that sound as real as they look. The result is a more believable and immersive experience for users that keeps them engaged and present within their environment.

These new capabilities deliver room acoustic simulation by accurately modeling reflections and reverberations, which are essential for creating a sense of space and depth. When sound waves bounce off surfaces and return to the listener, it can provide important spatial cues about the size and shape of the environment.

Our acoustics features can handle arbitrarily complex geometry, ensuring that even the most intricate environments are accurately simulated. Whether your VR scene is a winding cave, a bustling cityscape, or an intricate indoor environment, our technology can manage the complexity without compromising on performance.

Sound reflections are a core aspect of authentic audio experiences, but our technology goes a step further to enable sound that adapts dynamically in real time. It can simulate advanced propagation phenomena such as diffraction, occlusion, and obstruction. Diffraction allows sound to bend around obstacles, occlusion models how objects block sound, and obstruction simulates how sound changes when it passes through different materials. These phenomena are crucial for a realistic audio experience, especially in complex environments. The underlying technology is very similar to the approach used in ray-traced graphics rendering.

In the physical world, different materials interact with sound in unique ways. We’ve engineered Acoustic Ray Tracing to replicate this authenticity through the ability to represent the acoustic properties of various materials, such as brick, carpet, glass, and wood. This means that a carpeted living room will sound very different from a concrete warehouse, adding another layer of realism to the audio experience.

One of the standout benefits of our new acoustics features is their performance on mobile hardware. While other solutions in the market require powerful PCs due to their high performance cost, our SDK is optimized to run efficiently on mobile devices such as Quest headsets. This opens up new possibilities for high-quality audio simulation in mobile applications, making immersive audio more accessible than ever before.

Accurate acoustics can create situational awareness and add tactical elements to competitive games. For example, users can hear the footsteps of their opponents echoing down the hall and around the corner, giving them a clear sense of position even without line of sight, allowing them to sneak into a corner and set up an ambush.

For social and multiplayer apps, accurate acoustics (along with spatial audio) applied to speech can really enhance the feeling that other people are really in the environment with you. Acoustic Ray Tracing enables speech to adapt according to objects and space, deepening the sense of presence and togetherness in these experiences.

We understand that ease of integration is crucial for developers. That's why our acoustics features are designed to work seamlessly with existing workflows in Unity and Unreal game engines. Additionally, we support popular audio middleware solutions (FMOD and Wwise), making it easier to incorporate our technology into your projects regardless of how you like to work.

To help you get started, we’ve provide complete documentation (Unity | Unreal) that covers everything from basic setup to advanced features. Additionally, we are developing sample projects to serve as practical examples and best practices for integrating these new acoustics features into your own apps.

These advanced Acoustic Ray Tracing features are already being used to create more immersive experiences in upcoming AAA games such as Batman: Arkham Shadow, developed by Camouflaj and slated for release in late 2024. Collaborating with real-world VR game developers has allowed us to enhance the robustness and quality of our technology, offering a feature set tailored to meet their needs. We look forward to introducing Quest gamers to our Acoustic Ray Tracing in the world of Gotham City, and we can’t wait to see how you leverage these capabilities to drive innovation in your own projects moving forward.

Acoustic Ray Tracing features offer you the tools you need to create immersive audio experiences that feel nearly indistinguishable from the physical world. From simulating room acoustics and advanced propagation phenomena to handling complex geometry and accurately representing material properties, these features provide a comprehensive solution for realistic audio in VR. With seamless integration, complete documentation, and efficient performance on mobile hardware, you have everything you need to achieve new levels of immersion and push the boundaries of immersive audio.

Get started today by downloading Audio SDK (Unity | Unreal) and visiting the documentation for Acoustic Ray Tracing (Unity | Unreal). To stay up to date with the latest launches and news, be sure to follow us on X and Facebook, and subscribe to our monthly newsletter in your Developer Dashboard settings.