Develop
Develop
Select your platform

Interaction SDK Architecture Overview

If you haven’t worked with a tool like Interaction SDK before, there are many concepts and patterns that will be new to you. This high-level overview will explain what happens before, during, and after an interaction (which is an action, like a grab or teleport) so you’re aware of the concepts and patterns you’ll encounter in the SDK.

Stage 1: Input data arrives from headset

Your input method (hands, controllers, or both) rely on tracking data from your headset’s cameras and controllers to interact with the virtual environment. Although this is listed as Stage 1, Interaction SDK is always processing input.
Data flowing from the headset to the controllers and hands
Data flowing from the headset to the controllers and hands.

Stage 2: Interactor looks for interactable

After getting the data from your headset, Interaction SDK knows where your hand or controller is positioned, so as you move your hand or controller around, the interactor attached to it will search the scene for a matching interactable. Each interactor only looks for interactables that match its type (ex. a Grab interactor will only look for a Grab interactable). To learn how to add interactors to your hand or controller, see the Tutorials section in the sidebar.
Interactors on the active input method (controllers or hands) looking for matching interactables
Interactors on the active input method (controllers or hands) looking for matching interactables.

Stage 3: Interactor detects an interactable

Eventually, the interactor will detect an interactable using its ComputeCandidate() (and ComputeCandidateTiebreaker()) if needed). When the interactable is detected, a Pointer Event fires. This stage is the Hover state in the Interaction Lifecycle. For an interactor to detect a matching interactable, a couple of conditions must be true:
  • If you’re using hands, your hands must match that action’s unique Pose (ex. For a poke, the pointer finger is extended).
  • If you’re using controllers, you must be partially pressing the button that performs that action (ex. For a grab, you partially pull the trigger).
  • Your hand or controller are close enough to the interactable.
  • No other Interactors on the same hand or controller are currently selecting. This is because only one Interactor per hand can be selecting at a time.
If those conditions are true and there’s only one interactor on your hand or controller, then the interactor enters the Hover state.
A ray Interactor on the active input method (controllers or hands) finding a matching interactable
A ray interactor on the active input method (controllers or hands) finding a matching interactable.

Stage 4: Interactor selects interactable

Once an interactor is in the Hover state, it can go to the Select state by completing the action. For example, poking a button or grabbing an object instead of just hovering over it.

Stage 5: Selected interactable moves

Although the interactor is in the Select state, it doesn’t cause the interactable to move. Movement is handled behind the scenes by other components, like IsdkGrabbable.

Stage 6: Interactor deselects interactable

Once you release the object, the interactor transitions back to a Hover state and then the Normal (default) state.

Learn more

Now that you know how the interaction lifecycle in Interaction SDK, continue on to the following guides:
Did you find this page helpful?
Thumbs up icon
Thumbs down icon
Nav Logo
Virtual reality / Meta Horizon
Virtual reality / Meta Horizon
Virtual reality / Meta Horizon
English (US)
© 2025 Meta