Develop

How to add gaze interactions with Spatial SDK

Updated: Sep 4, 2026

Overview

Gaze interactions use eye tracking to aim and a hand pinch to act. Users can select UI, scroll panels, adjust sliders, and grab objects.

Gaze activates when no controllers are tracked. Tracked controllers take priority. Near-field hand interactions also suppress gaze to prevent competing input.

UiUnderstandingFeature identifies interactive elements inside panels. This data improves targeting and provides element bounds and shapes for hover effects.

Without UI Understanding, gaze can still target complete panels and 3D objects. It cannot target individual elements inside a panel.

Before you begin

Create a Spatial SDK project. See Create a Spatial SDK project.
Use a headset with eye-tracking hardware for eye-driven targeting.
Enable eye tracking under Settings > Movement Tracking > Eye Tracking.

Configure gaze interactions

Declare permissions

Add the eye-tracking and hand-tracking declarations to AndroidManifest.xml. Gaze uses hand pinch for actuation.

<uses-feature android:name="oculus.software.eye_tracking" android:required="true" />
<uses-permission android:name="horizonos.permission.EYE_TRACKING" />

<uses-feature android:name="oculus.software.handtracking" android:required="false" />
<uses-permission android:name="horizonos.permission.HAND_TRACKING" />

If your app requires gaze input, set oculus.software.eye_tracking to required.

If your app can use controllers without tracked hands, keep oculus.software.handtracking optional.

Request the eye-tracking permission at runtime before gaze input is required:

import android.content.pm.PackageManager
import androidx.core.app.ActivityCompat

val permission = "horizonos.permission.EYE_TRACKING"
if (checkSelfPermission(permission) != PackageManager.PERMISSION_GRANTED) {
    ActivityCompat.requestPermissions(this, arrayOf(permission), 0)
}

Register the features

override fun registerFeatures(): List<SpatialFeature> {
    return listOf(
        VRFeature(this),
        UiUnderstandingFeature(this),
    )
}

VRFeature uses VrInputSystemType.INTERACTION_SDK by default. This input system provides gaze, poke, grab, and ray interactions.

UiUnderstandingFeature provides element-level targeting and hover effects. It also registers the bridge for cross-process panels.

Note: VrInputSystemType.SIMPLE_CONTROLLER does not register ISDK. Gaze, poke, grab, and ISDK ray interactions are unavailable in this mode.

Configure gaze at runtime

Get IsdkSystem from the activity system manager:

import com.meta.spatial.isdk.IsdkSystem

val isdkSystem = systemManager.findSystem<IsdkSystem>()

Use gazeEnabled to control gaze without changing other ISDK interactions:

isdkSystem.gazeEnabled = false // Disable gaze
isdkSystem.gazeEnabled = true // Re-enable gaze

gazeEnabled defaults to true.

Use active to control all ISDK input:

isdkSystem.active = false // Disable input
isdkSystem.active = true // Re-enable input

Configure pointer reach

The default maximum distance for scene pointer interactions is 5 meters.

If interactive content is farther away, set a longer distance:

isdkSystem.setScenePointerDistance(10.0f)

Read the current value with getScenePointerDistance().

Pause UI Understanding

When the app must stop collecting panel elements, disable UI Understanding. Enable it to resume collection:

import com.meta.spatial.uiunderstanding.UiUnderstandingSystem

val uiUnderstandingSystem = systemManager.findSystem<UiUnderstandingSystem>()
uiUnderstandingSystem.disable()  // Stops per-panel element harvesting
uiUnderstandingSystem.enable()   // Resume per-panel element harvesting

Configure HMD gaze fallback

If eye-gaze data is unavailable, enable HMD gaze fallback to use the headset forward direction:

isdkSystem.hmdGazeFallbackEnabled = true

hmdGazeFallbackEnabled defaults to false.

Handle pointer events

isdkSystem.registerObserver { pointerEvent ->
    // Handle the pointer event.
}

Use registerInteractableObserver for one entity:

val handle = isdkSystem.registerInteractableObserver(entity) { pointerEvent ->
    // Handle the pointer event for this entity.
}

Use the returned InteractableObserverHandle to unregister the observer.

Gaze hover events are suppressed by default. This prevents duplicate hover feedback from the system and the app.

If your app renders custom hover feedback, disable suppression:

isdkSystem.gazeHoverSuppressionEnabled = false

Selection, scroll, grab, and other actuating events remain available when suppression is enabled.

Use debug visualization

Enable ISDK debug tools during development:

isdkSystem.debugToolsEnabled = true

The debug tools show interaction rays, targets, and hit points. They apply to all ISDK interactions and have a performance cost.

Disable the debug tools before you release the app.

To show the bounds collected by UI Understanding, enable its separate visualization:

import com.meta.spatial.uiunderstanding.UiUnderstandingSystem

val uiUnderstandingSystem = systemManager.findSystem<UiUnderstandingSystem>()
uiUnderstandingSystem.setVisualizationEnabled(true)

Configure hover effects

GazeHoverEffectSystem renders hover feedback for targeted UI elements.

Enable or disable hover effects

Get the system. Then enable or disable hover effects for the current experience state:

import com.meta.spatial.isdk.GazeHoverEffectSystem

val hoverSystem = systemManager.findSystem<GazeHoverEffectSystem>()
hoverSystem.disable()
hoverSystem.enable()

Select a rendering mode

GazeHoverEffectSystem.mode accepts two values:

GazeHoverEffectMode.System uses the OS-composited hover effect when a panel supports it.
GazeHoverEffectMode.View applies the effect to the Android panel content.

System is the default. The System mode yields hover effects that most closely resemble those used elsewhere in the Quest ecosystem.

If a panel cannot use the system path, that panel uses the View path automatically.

The view path works with every panel type. It requires Android API 33 for RuntimeShader and RenderEffect.

If you need one rendering path across all panels, select the view path:

import com.meta.spatial.isdk.hovereffects.GazeHoverEffectMode

hoverSystem.mode = GazeHoverEffectMode.View

A mode change clears the current effect when the system applies the new mode.

Customize the hover shape

For Android Views, the system reads the shape from the ViewOutlineProvider.

If the default outline does not match the visible element, set a custom outline:

import android.graphics.Outline
import android.view.View
import android.view.ViewOutlineProvider

view.outlineProvider = object : ViewOutlineProvider() {
    override fun getOutline(view: View, outline: Outline) {
        outline.setRoundRect(0, 0, view.width, view.height, view.height / 2f)
    }
}
view.clipToOutline = true

Use outline.setOval(...) for circles. Use outline.setPath(path) for a custom path.

For Jetpack Compose, use standard shape modifiers. The system preserves these shapes for in-process and cross-process panels.

import androidx.compose.foundation.background
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.MaterialTheme
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp

Box(
    modifier = Modifier
        .size(width = 120.dp, height = 40.dp)
        .background(
            color = MaterialTheme.colorScheme.primary,
            shape = RoundedCornerShape(percent = 50),
        )
        .clickable { },
)

Use CircleShape for circles. Use GenericShape for a custom Compose path.

Support sliders

Gaze pinch uses accessibility range semantics to adjust sliders. The SDK sends ACTION_SET_PROGRESS updates during the gesture.

Standard Android SeekBar and Material Compose Slider controls provide the required semantics out of the box.

A custom Android View must expose all of these properties through accessibility:

A finite minimum, maximum, and current value.
An enabled and visible accessibility node.
Support for AccessibilityNodeInfo.ACTION_SET_PROGRESS.

For a custom Compose control, publish the range and implement the progress action:

import androidx.compose.foundation.progressSemantics
import androidx.compose.ui.semantics.semantics
import androidx.compose.ui.semantics.setProgress

modifier
    .semantics {
        setProgress { requestedValue ->
            onValueChange(requestedValue.coerceIn(valueRange))
            true
        }
    }
    .progressSemantics(value, valueRange)

If a control advertises a slider without a valid progress action, the SDK consumes the gaze gesture without changing the value.

Enable gaze on cross-process panels

A panel activity can run in a process that is separate from the main Spatial SDK activity.

The main process cannot read that activity View hierarchy directly. Enable the cross-process UI Understanding bridge in the panel activity.

Register UiUnderstandingFeature in the main activity.
Call CrossProcessUiUnderstanding.enable(this) after setContentView() in each cross-process panel activity.

import com.meta.spatial.uiunderstanding.crossprocess.CrossProcessUiUnderstanding

class MyCrossProcessPanelActivity : Activity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.my_layout)
        CrossProcessUiUnderstanding.enable(this)
    }
}

This call enables element targeting, hover effects, and slider actions for the panel.

Gaze interaction behavior

The system manages input transitions automatically:

Gaze activation: Gaze is active when no controllers are tracked and the hands are visible.
Controller priority: Tracked controllers disable gaze and take priority.
Near-field suppression: Direct hand interactions suppress gaze to prevent simultaneous targeting.
Selection suppression: An active gaze selection clears the hover effect.

Gaze supports these gestures:

Select: Look at an interactive element and pinch with either hand.
Scroll: Look at scrollable content, pinch, and move the pinching hand.
Adjust a slider: Look at a slider, pinch, and move the pinching hand.
Microgesture scroll: Move the thumb along the index finger to scroll the targeted panel.
Grab and move: Look at a Grabbable entity, pinch, and move the pinching hand.

Troubleshooting

Gaze does not follow eye movement

Declare horizonos.permission.EYE_TRACKING in the manifest.
Request the permission at runtime.
Enable eye tracking in the device settings.
Set IsdkSystem.gazeEnabled to true.

Pinch does not select or grab

Declare horizonos.permission.HAND_TRACKING in the manifest.
Declare oculus.software.handtracking in the manifest.
Keep the hands visible to the headset cameras.

Hover does not appear on UI elements

Register UiUnderstandingFeature.
Call GazeHoverEffectSystem.enable().
Give the element interactive semantics.
If you select View mode, use Android API 33 or later.

See Provide UI understanding for Look and Pinch for interactability requirements.

A hover shape is incorrect

For Android Views, set the correct ViewOutlineProvider. For Compose, use clipping if the shape modifier requires it.

See Customize the hover shape for examples.

A slider does not change value

Publish valid range information and support ACTION_SET_PROGRESS.

A cross-process panel does not respond

Register UiUnderstandingFeature in the main activity.
Call CrossProcessUiUnderstanding.enable(this) after setContentView() in the panel activity.

An observer does not receive gaze hover events

Gaze hover events are suppressed by default. If the app needs these events, set gazeHoverSuppressionEnabled to false.

Actuating events remain available while hover suppression is enabled.

Gaze stops when controllers are tracked

This behavior is expected. Tracked controllers take priority, and gaze returns when controller tracking stops.

Provide UI understanding for Look and Pinch: How the system identifies interactable elements and infers element shapes
Look and Pinch for 2D apps: Overview of the gaze privacy model and targeting concepts for 2D panel apps
Spatial SDK samples⁠
Create a Spatial SDK project: Initial project setup and configuration for Spatial SDK
Learn Spatial SDK VR concepts: Key concepts for VR development with Spatial SDK