Brightness Estimation sample overview

Updated: May 7, 2026

Overview

This sample demonstrates how to estimate ambient room brightness from passthrough camera pixel data and adapt your mixed reality experience based on lighting conditions. It covers real-time camera frame access, luminance computation, smoothing techniques to prevent flickering, and threshold-based UI changes.

View the sample source code on GitHub⁠.

What you will learn

Access passthrough camera frame data using PassthroughCameraAccess from the Mixed Reality Utility Kit (MRUK)

Compute perceived brightness using ITU-R BT.709 luminance coefficients

Implement a rolling average buffer to smooth brightness readings over time

Create a threshold-based response system with hysteresis to prevent state flickering

Request and handle passthrough camera permissions at runtime

Requirements

Meta Quest 3 or Quest 3S

For complete development environment setup, Unity version requirements, and SDK dependencies, see Set up your development environment⁠. For project-specific build prerequisites, see the sample repository README.

Get started

Clone the Unity-PassthroughCameraApiSamples repository⁠ from GitHub. Open the project in Unity, navigate to Assets/PassthroughCameraApiSamples/BrightnessEstimation/, and open the BrightnessEstimation.unity scene. Build and deploy to your Quest device following the instructions in the repository README. When you run the sample, you see banners that update based on detected room brightness.

Explore the sample

The sample uses a producer-consumer architecture with two key scripts and MRUK’s passthrough camera access component.

File / Scene	What it demonstrates	Key concepts
BrightnessEstimation.unity	Scene hierarchy with camera rig, passthrough layer, and brightness UI	Integration of `MRUK` camera access with Unity scene structure
BrightnessEstimationManager.cs	Per-frame luminance computation and smoothing	PassthroughCameraAccess API, rolling average buffer, UnityEvent broadcasting
BrightnessEstimationDebugger.cs	Threshold-based state machine with hysteresis	Event-driven UI updates, state tracking to prevent flickering
RequestPermissionsOnce.cs	Auto-request permissions at startup	Runtime permission handling with `OVRPermissionsRequester`
PassthroughCameraAccessPrefab	`MRUK` camera component configured to 640x480 resolution	Camera initialization and resolution override
BrightnessEstimationBannersOn / BrightnessEstimationBannersOff	Emissive 3D text materials (yellow/purple)	Context-aware visual feedback

Runtime behavior

When you run the sample, the system continuously analyzes passthrough camera frames to compute ambient brightness. Floating 3D text banners appear in front of you with emissive yellow and purple materials. When the room is too dark (brightness at or below the lower threshold), you see “IS TOO DARK, TURN LIGHTS ON!” with the “lights on” banners visible. When the room is too bright (brightness at or above the upper threshold), the message changes to “TOO BRIGHT, TURN LIGHTS OFF!” with the “lights off” banners visible. Values between the two thresholds maintain the previous state without triggering changes, preventing rapid flickering as lighting conditions fluctuate slightly.

Key concepts

Accessing passthrough camera frames

The sample uses PassthroughCameraAccess from MRUK to retrieve real-time camera data. Notice how BrightnessEstimationManager guards its processing behind a camera readiness check each frame:

if (m_cameraAccess.IsPlaying)
{
    m_debugger.text = GetRoomAmbientLight();
    m_onBrightnessChange?.Invoke(GetGlobalBrightnessLevel());
}

When IsPlaying returns true, the sample calls m_cameraAccess.GetColors() to retrieve an array of pixel color values and reads m_cameraAccess.CurrentResolution to determine the frame dimensions. When the camera is not yet active, it displays permission status text instead. The PassthroughCameraAccess component is wired via [SerializeField] in the Unity Inspector. For API details, see the Passthrough Camera API getting started guide.

Computing perceived brightness

The sample converts RGB pixel data to luminance using the standard ITU-R BT.709 formula. The producer script iterates through all pixels and accumulates weighted color values, then normalizes by total pixel count to produce a per-frame brightness value. The default thresholds (10 and 50) are configurable in the Unity Inspector to match your application’s needs. For the complete implementation, see BrightnessEstimationManager.GetRoomAmbientLight().

Smoothing with a rolling average buffer

Raw per-frame brightness values fluctuate due to camera noise and minor lighting changes. The sample maintains a FIFO buffer (default size 10 frames, configurable 1-100) and computes the arithmetic mean of buffered values before broadcasting events. The manager limits refresh rate to 0.05 seconds (20 Hz) to balance responsiveness with CPU cost. This pattern prevents jittery UI updates while maintaining sufficient responsiveness for typical lighting changes. See BrightnessEstimationManager for the buffer implementation.

Threshold response with hysteresis

The consumer script uses a two-state system (dark or bright) with configurable thresholds (m_minBrightnessLevel default 10, m_maxBrightnessLevel default 50). State changes only occur when brightness crosses a threshold AND the current state differs from the target state. Values between the two thresholds form a hysteresis band — no state change fires, which prevents flickering when brightness hovers near a boundary. The sample tracks state using an integer flag and fires separate UnityEvents (m_onTooDark, m_onTooLight) that toggle banner GameObjects. For implementation details, see BrightnessEstimationDebugger.OnChangeBrightness().

Runtime permission handling

The sample requests passthrough camera permissions automatically using [RuntimeInitializeOnLoadMethod] to ensure permissions are requested once per session before any scene loads. The sample requests both OVRPermissionsRequester.Permission.Scene and OVRPermissionsRequester.Permission.PassthroughCameraAccess. The manager displays permission status text when the camera is not yet playing. See RequestPermissionsOnce.cs for the auto-request pattern.

Extend the sample

Add granular brightness zones: Extend the two-threshold system in BrightnessEstimationDebugger.cs to support multiple brightness bands (very dark, dark, normal, bright, very bright) with different visual responses for each zone, creating more nuanced environmental adaptation

Implement time-based averaging: Modify the rolling buffer in BrightnessEstimationManager.cs to use weighted averaging that favors recent frames over older ones, or add exponential smoothing for faster response to rapid lighting changes while still filtering noise

Combine with other samples: Use BrightnessEstimationManager’s m_onBrightnessChange event to feed brightness data into the MultiObjectDetection sample for adjusting ML inference parameters based on lighting, or pair it with CameraToWorld to place virtual lights that respond to real-world brightness

Passthrough Camera API overview⁠

Passthrough Camera API getting started guide⁠

CameraViewer sample — simplest camera access example, good starting point⁠

MultiObjectDetection sample — ML inference on camera data⁠

Unity-PassthroughCameraApiSamples GitHub repository⁠