Get started with Occlusions

Follow the instructions in Passthrough API Overview to set up your project.

In addition to enabling passthrough in your project settings, you must also enable Scene Support. To do this, go to Edit > Project Settings > MetaXR, and under Mobile, check Scene Support.

You will need to request Runtime permissions for Scene. This is done when you first start environment depth, however, if you want more control over when the dialog is shown, this can be done using the Request Android Permissions Blueprint node with the string com.oculus.permission.USE_SCENE.

In order to use the Depth API, you must start capturing depth information into a swap chain at runtime. You can do this by using the StartEnvironmentDepth Blueprint node or calling FOculusXRHMD::StartEnvironmentDepth from C++. When it is no longer needed, call StopEnvironmentDepth to avoid using unnecessary resources.

The main use case for the Depth API is allowing real-world objects to occlude virtual objects visible in passthrough. There are two types of occlusion available: hard occlusions and soft occlusions. Hard occlusions are easier to integrate and perform better than soft occlusions, but soft occlusions are more visually appealing. In either case, the passthrough layer should be configured as Underlay and Reconstructed.

Hard occlusions can be enabled via the SetXROcclusionsMode Blueprint node by setting the value to HardOcclusions. This will insert a draw call after opaque objects in the scene have been rendered. That will render the environment depth into the scene’s depth buffer and punch through areas that should be occluded by passthrough.

In order to enable soft occlusions, you will need to use the Meta fork of the Unreal Engine. This includes changes to the shaders that are not present in Epic’s standard Unreal Engine.

Soft occlusions require Support XR Soft Occlusions to be enabled in Project Settings. Click Edit > Project Settings, select Engine > Rendering on the left and navigate to the VR section to find the checkbox (alternatively, you can search for it using the search bar). Enabling this will double the number of shader permutations compiled.

Note: Only enable this setting if you plan to use soft occlusions at runtime.

Once this is done, soft occlusions can be enabled at runtime via the SetXROcclusionsMode Blueprint node by setting the value to SoftOcclusions. This works by adding some additional instructions to the pixel shader of all materials to sample the environment depth and compare it with the depth of the pixel being rendered. Materials can be omitted from this process by disabling the XR Soft Occlusions checkbox on the material. This can be used in cases where certain materials should never be occluded, such as a user interface for example.

A common issue with XR occlusions is depth fighting with environment when a thin virtual object is placed close/against a wall or floor. A depth bias can be used to mitigate this and give priority to the virtual object so that it doesn’t get occluded when it shouldn’t.

The XR Soft Occlusions Depth Bias parameter on each material can be adjusted to control this. The linear depth of the virtual object will be adjusted such that biased linear depth = linear depth - linear depth * bias before comparing to the depth of the environment. Use a small positive value (e.g. 0.06) to give a slight bias to the virtual objects.