Chapter 15: Depth Occlusion

• Set up depth sensing for your AR experience.
• Apply depth occlusion to virtual objects so they hide behind real-world geometry.
• Choose between occlusion modes for different quality/performance tradeoffs.
• Configure occlusion parameters like blur radius.
• Access raw depth data for advanced use cases.

• DepthSensingSystem - Core system that retrieves depth data from the WebXR session and injects occlusion shaders into entity materials.
• DepthOccludable - Component that marks an entity for occlusion by real-world geometry.
• OcclusionShadersMode - Enum for selecting the occlusion algorithm (Soft, Hard, or MinMax).

import {
  World,
  SessionMode,
  ReferenceSpaceType,
  DepthSensingSystem,
  DepthOccludable,
  Mesh,
  SphereGeometry,
  MeshStandardMaterial,
} from '@iwsdk/core';

World.create(document.getElementById('scene-container'), {
  xr: {
    sessionMode: SessionMode.ImmersiveAR,
    referenceSpace: ReferenceSpaceType.Unbounded,
    features: {
      depthSensing: {
        required: true,
        usage: 'gpu-optimized',
        format: 'float32',
      },
      anchors: { required: true },
      unbounded: { required: true },
    },
  },
}).then((world) => {
  scene.background = null; // Transparent background for AR

  // Register the depth sensing system
  world.registerSystem(DepthSensingSystem).registerComponent(DepthOccludable);

  // Create a virtual sphere
  const sphere = new Mesh(
    new SphereGeometry(0.2),
    new MeshStandardMaterial({ color: 0xff4444, transparent: true }),
  );
  sphere.position.set(0, 1.0, -1.0);
  world.scene.add(sphere);

  // Mark it as occludable — it will now hide behind real-world surfaces
  const entity = world.createTransformEntity(sphere);
  entity.addComponent(DepthOccludable);
});

World.create(container, {
  xr: {
    sessionMode: SessionMode.ImmersiveAR,
    referenceSpace: ReferenceSpaceType.Unbounded,
    features: {
      depthSensing: {
        required: true,
        usage: 'gpu-optimized', // or 'cpu-optimized'
        format: 'float32', // or 'luminance-alpha'
      },
      anchors: { required: true },
      unbounded: { required: true },
    },
  },
});

world
  .registerSystem(DepthSensingSystem, {
    configData: {
      enableOcclusion: true, // Master toggle for occlusion rendering
      enableDepthTexture: true, // Create/update depth textures each frame
      useFloat32: true, // Use Float32 depth data (higher precision)
      blurRadius: 20.0, // Blur radius in pixels for SoftOcclusion mode
    },
  })
  .registerComponent(DepthOccludable);

const entity = world.createTransformEntity(mesh);
entity.addComponent(DepthOccludable);

• mode - Occlusion algorithm: OcclusionShadersMode.SoftOcclusion (default), HardOcclusion, or MinMaxSoftOcclusion.

// Default — soft occlusion with smooth edges
entity.addComponent(DepthOccludable);

// Hard occlusion — sharp edges, best performance
entity.addComponent(DepthOccludable, {
  mode: OcclusionShadersMode.HardOcclusion,
});

// MinMax soft occlusion — highest quality with performance cost, edge-aware blending
entity.addComponent(DepthOccludable, {
  mode: OcclusionShadersMode.MinMaxSoftOcclusion,
});

• SoftOcclusion is the default and works well for most cases. It uses a 13-tap sampling pattern (center, inner ring, and outer ring) controlled by the blurRadius config.
• HardOcclusion does a single depth lookup per fragment. It’s the cheapest option but produces hard edges that can look aliased.
• MinMaxSoftOcclusion runs a preprocessing pass that clusters a 4×4 depth neighborhood into near/far groups, then uses edge-aware interpolation at render time. This produces the smoothest results at depth discontinuities (for example, an object edge against a distant background) at the cost of an additional render pass.

• enableOcclusion - Master toggle for occlusion rendering (default: true).
• enableDepthTexture - Whether to update depth textures each frame (default: true).
• useFloat32 - Use Float32 precision for CPU depth data (default: true).
• blurRadius - Blur radius in pixels for SoftOcclusion mode (default: 20.0).

• cpuDepthData - Array of XRCPUDepthInformation objects (populated in CPU-optimized mode).
• gpuDepthData - Array of XRWebGLDepthInformation objects (populated in GPU-optimized mode).
• rawValueToMeters - Conversion factor from raw depth values to meters.

class MyDepthSystem extends createSystem() {
  update() {
    const depthSystem = this.world.getSystem(DepthSensingSystem);

    // Access CPU depth data (when using cpu-optimized mode)
    if (depthSystem.cpuDepthData.length > 0) {
      const depthInfo = depthSystem.cpuDepthData[0];
      const distanceMeters = depthInfo.getDepthInMeters(0.5, 0.5); // Center of screen
      console.log('Center depth:', distanceMeters, 'meters');
    }
  }
}

import {
  DepthOccludable,
  Mesh,
  BoxGeometry,
  MeshStandardMaterial,
  Color,
} from '@iwsdk/core';

const cube = new Mesh(
  new BoxGeometry(0.2, 0.2, 0.2),
  new MeshStandardMaterial({
    color: new Color(0x44ff44),
    transparent: true, // Required for occlusion
    metalness: 0.3,
    roughness: 0.4,
  }),
);
cube.position.set(0, 1.0, -0.8);
scene.add(cube);

const entity = world.createTransformEntity(cube);
entity.addComponent(DepthOccludable);

import {
  AssetManager,
  DepthOccludable,
  OcclusionShadersMode,
} from '@iwsdk/core';

const { scene: robotMesh } = AssetManager.getGLTF('robot');
robotMesh.position.set(0.6, 0, -1.0);
scene.add(robotMesh);

const entity = world.createTransformEntity(robotMesh);
entity.addComponent(DepthOccludable, {
  mode: OcclusionShadersMode.MinMaxSoftOcclusion,
});

import {
  DepthOccludable,
  DistanceGrabbable,
  Interactable,
  MovementMode,
  XRAnchor,
} from '@iwsdk/core';

const entity = world.createTransformEntity(mesh);
entity.addComponent(Interactable);
entity.addComponent(DistanceGrabbable, {
  movementMode: MovementMode.MoveFromTarget,
});
entity.addComponent(XRAnchor);
entity.addComponent(DepthOccludable);

const depthSystem = world.getSystem(DepthSensingSystem);

// Disable occlusion
depthSystem.config.enableOcclusion = false;

// Re-enable occlusion
depthSystem.config.enableOcclusion = true;

const depthSystem = world.getSystem(DepthSensingSystem);

// Tighter blur — edges closer to hard occlusion
depthSystem.config.blurRadius = 5.0;

// Wider blur — smoother, more forgiving edges
depthSystem.config.blurRadius = 40.0;

• Verify depthSensing is included in your XR features config with required: true.
• Ensure the device supports the WebXR depth-sensing module.
• Check that you’re in an AR session (SessionMode.ImmersiveAR), not VR.
• Confirm DepthSensingSystem is registered and enableOcclusion is true.

• Make sure the entity has the DepthOccludable component.
• The DepthOccludable component must be added after the entity’s object3D has its final meshes. If you load a GLTF asynchronously, add the component after the model is ready.

• For aliased/jagged edges, switch from HardOcclusion to SoftOcclusion.
• For bleeding at depth edges (for example, occlusion leaks around object boundaries), try MinMaxSoftOcclusion.
• Adjust blurRadius — lower values produce tighter edges, higher values produce smoother but potentially less accurate edges.

• The occlusion system injects GLSL code into the material’s vertex and fragment shaders via onBeforeCompile. Custom shaders that don’t use the standard Three.js shader structure may not be compatible.
• Ensure your shader includes the #include <output_fragment> chunk, which is where the occlusion alpha is applied.

1. Choose the right occlusion mode — HardOcclusion is cheapest, SoftOcclusion is a good default, and MinMaxSoftOcclusion adds an extra render pass per eye. Use MinMaxSoftOcclusion only where edge quality matters most.
2. Prefer gpu-optimized depth — GPU-optimized depth sensing keeps depth data on the GPU as a texture array, avoiding CPU read-back overhead. Use cpu-optimized only when you need to read depth values in JavaScript.
3. Limit entities with occlusion — Each entity with DepthOccludable has its shader uniforms updated every frame. Only mark entities that need occlusion.

1. Use SoftOcclusion as the default mode — it provides a good balance of quality and performance.
2. Reserve MinMaxSoftOcclusion for hero objects where edge quality is critical.
3. Use HardOcclusion for small or distant objects where edge quality is less noticeable.
4. Test occlusion with various real-world scenes — flat surfaces, object edges, and corners all stress occlusion differently.
5. Combine depth occlusion with Scene Understanding (Chapter 11) and Environment Raycast (Chapter 14) for fully grounded AR experiences.

• examples/depth-occlusion - AR scene with multiple occlusion modes, grabbable objects, and GLTF models.

cd immersive-web-sdk
pnpm install
pnpm run build:tgz
cd examples/depth-occlusion
npm install
npm run dev

• Using depth data directly for proximity-based effects (for example, particles that react to nearby surfaces).
• Combining occlusion with physics so objects can both collide with and hide behind real-world geometry.
• Building AR product visualization apps where items sit on and hide behind real furniture.