Develop

Interaction SDK Architecture Overview

Updated: Aug 31, 2026

If you haven’t worked with a tool like Interaction SDK before, there are many concepts and patterns that will be new to you. This high-level overview will explain what happens before, during, and after an interaction (which is an action, like a grab or teleport) so you’re aware of the concepts and patterns you’ll encounter in the SDK.

Stage 1: Input data arrives from headset

Your input method (hands, controllers, or both) relies on tracking data from your headset’s cameras and controllers to interact with the virtual environment. Although this is listed as Stage 1, Interaction SDK is always processing input.

Data flowing from the headset to the controllers and hands.

Stage 2: Interactor looks for interactable

After getting the data from your headset, Interaction SDK knows where your hand or controller is positioned, so as you move your hand or controller around, the interactor attached to it will search the scene for a matching interactable. Each interactor only looks for interactables that match its type (ex. a Grab interactor will only look for a Grab interactable). To learn how to add interactors to your hand or controller, see the Tutorials section in the sidebar.

Interactors on the active input method (controllers or hands) looking for matching interactables.

Stage 3: Interactor detects an interactable

Eventually, the interactor will detect a matching interactable through its internal candidate detection system. When the interactable is detected, a Pointer Event fires. This stage is the Hover state in the Interaction Lifecycle. For an interactor to detect a matching interactable, a couple of conditions must be true:

If you’re using hands, your hands must match that action’s unique Pose (for example, for a poke, the pointer finger is extended).
If you’re using controllers, you must be partially pressing the button that performs that action (for example, for a grab, you partially pull the trigger).
Your hand or controller are close enough to the interactable.
No other interactors on the same hand or controller are currently selecting. This is because only one interactor per hand can be selecting at a time.

If those conditions are true, the interactor enters the Hover state.

A ray interactor on the active input method (controllers or hands) finding a matching interactable.

Stage 4: Interactor selects interactable

Once an interactor is in the Hover state, it can go to the Select state by completing the action—for example, poking a button or grabbing an object rather than only hovering over it.

Stage 5: Selected interactable moves

Although the interactor is in the Select state, it doesn’t cause the interactable to move. Movement is handled behind the scenes by other components, like IsdkGrabbableComponent.

Stage 6: Interactor deselects interactable

Once you release the object, the interactor transitions back to a Hover state and then the Normal (default) state.

The Gaze Rig

Gaze uses the same hover-and-select model described in Stages 1 through 6, but it is driven by an optional rig that sits alongside the per-hand rigs (the per-hand containers that hold the ray, poke, and grab interactors).

The UIsdkGazeRigComponent is an optional component placed on the same pawn or tracking-space-root actor as the hand rigs. Its Blueprint display name is ISDK Gaze Rig Component, and it ships in the OculusInteractionPrebuilts module.

The gaze rig owns a UIsdkGazeTargetingComponent from the sibling IsdkGaze module, reachable via GetGazeTargeting(). That component resolves the current gaze pose to a scene target every frame.

The gaze rig also owns exactly one UIsdkRegionInteractionRigComponent, which in turn owns exactly one UIsdkRegionInteractor — the gaze-driven selection interactor — shared by both hands. This is an architectural asymmetry: ray, poke, and grab rigs are per-hand, but the region interactor is shared. Either hand can actuate it, and the first hand to begin a selection owns the interaction until release.

On BeginPlay, the gaze rig calls WireHandInteractorGazeGate(). That method inserts a UIsdkConditionalGroupNone (a NOR of the rig’s gaze-mode conditional) into each sibling ray-interaction rig’s enabled conditional group. Hand ray interactors are enabled only while gaze mode is off: gaze or hand rays, never both.

Gaze does not introduce a new interactable type. Existing UIsdkRayInteractable, UIsdkGrabbableComponent, and UIsdkHoverableComponent each expose a GazePolicy property of type EIsdkGazePolicy with values UsePluginSetting, Allow, and Disallow, so individual objects opt in or out per-instance.

Pawn / Tracking Space Root Actor
├── UIsdkGazeRigComponent
│   ├── UIsdkGazeTargetingComponent
│   └── UIsdkRegionInteractionRigComponent
│       └── UIsdkRegionInteractor  (shared, both hands)
├── UIsdkHandRigComponent  (left)
└── UIsdkHandRigComponent  (right)

See Hands, Controllers, and Gaze for how the gaze rig relates to the hand rigs, and Gaze Interactions for per-object configuration and Blueprint setup.

Learn more

Now that you know how the interaction lifecycle works in Interaction SDK, continue to the following guides:

Design guidelines

Design guidelines are Meta’s human interface standards and design frameworks that help you create safe, user-oriented, and retainable immersive and passthrough user experiences.

Core interactions

Input mappings: Understand how input mappings bridge modalities and interaction types.
Input hierarchy: Understand the input hierarchy.
Multimodality: Understand multimodality.
Ray casting: Understand indirect interaction through ray casting.
Touch: Understand direct interaction through touch.
Grab: Understand grab interactions for object manipulation.
Microgestures: Understand microgesture interactions.