Version 71 of the Meta XR Core SDK introduced the MRUK Raycast API in public beta. It is designed for applications that want to place a 2D panel or 3D object somewhere in front of the user with minimal setup and effort required. This can be helpful when placing a board game on a table or sticking a UI panel to a wall, for example. The requirements for this placement are minimal: The user should simply look in the direction of the placement. This enables the users to put on a headset in a room they’ve never been to before, and immediately start interacting with both 2D and 3D content.
This sample runs with OpenXR. Simply makes sure that, alongside the OpenXR plugin, you also install the Unity OpenXR Meta plugin extension. This is necessary for using the Depth API.
How it works
To place an object in a room, there is no need for colliders at all, instead the new EnvironmentRaycastManager is utilized, which heavily relies on the EnvironmentDepthManager. The basic concept of placing an object looks like the following:
EnvironmentRaycastManager.Raycast(new Ray(origin, direction), out var hitInfo)
Therefore, the most simple bare bones instant placement logic could look something like this:
using Meta.XR.MRUtilityKit;
using UnityEngine;
public class BasicInstantPlacement : MonoBehaviour
{
public Transform Object;
public EnvironmentRaycastManager RaycastManager;
void Update()
{
// Check if a surface below the object is hit
if (RaycastManager.Raycast(new Ray(Object.position, Vector3.down), out var hitInfo))
{
// Position the object on the hitpoint/detected surface
Object.position = hitInfo.point;
}
}
}
The important part here is the EnvironmentRaycastManager.Raycast method. It performs a raycast against the environment and returns a hit result with information such as the hit point, normal of the surface hit, and normal confidence. This is already all the information needed to detect a surface and place any object on that surface. The Raycast API, part of MRUK, is as easy to work with as Unity’s Physics.Raycast, which most Unity developers will already be familiar with.
Sample Scenes
The Depth Effects scene shows off various visual effects that can be achieved by utilizing the information from the EnvironmentDepthManager directly in the shaders. It contains an orb spawner, which shoots an orb in the direction the user is looking at. The orb has a shader applied that uses the depth information to occlude the orb behind real objects. Furthermore, when the orb explodes, it shoots a wave through the depth map of the space.
The Instant Content Placement scene demonstrates how to use Depth Raycasting to detect and place surfaces. There is also a separate shader to render a shadow below the object and cut it off whenever it extends a surface, such as a table, just like a real shadow. This allows developers to create a more realistic and immersive experience without having to manually scan their environment.
Custom Shaders
This MR Motif includes several sample shaders designed to assist developers in quickly creating custom occlusion shaders. For example, the DepthLookingGlassMotif shader visualizes depth maps on objects such as quads, displaying both physical and virtual object depth to enhance gameplay interactions. To visualize virtual objects accurately, ensure they use opaque shaders, as transparent objects do not write to Unity’s depth buffer. To achieve invisible objects that still affect depth, the DepthMaskMotif shader can be applied. Additionally, for virtual depth visualization, depth textures must be enabled in the camera’s rendering settings or the Universal Render Pipeline asset.
The Depth Effects scene demonstrates two additional visual effects. The DepthRelightingMotif shader applies a glow or lighting effect directly to the depth map, shown by the blue glow of the orb moving through the environment. The DepthScanEffectMotif shader is applied to an invisible sphere, expanding over time and coloring intersections with the depth map to produce a shockwave effect. The image below illustrates the ShaderGraph implementation of the Orb, providing an example of how occlusion can be integrated into existing shaders. Additional details are available in the Depth API GitHub repository.