Develop

Support Hands, Controllers, and Gaze on the Same Object

Updated: Sep 2, 2026

Experimental

This feature is considered experimental. Use caution when implementing it in your projects as it could have performance implications resulting in artifacts or other issues that may affect your project.

In this guide, you will learn how to reach hand tracking and gaze users from an interaction you built with controllers. It assumes you have a scene with the comprehensive interaction rig and at least one working interaction on an object.

There is no conversion step. The comprehensive rig already drives hands, controllers, and controller driven hands at the same time, and one interactable serves whichever device is active. Gaze is an addition on top of that rather than a replacement.

By the end of this guide, you should be able to:

Explain why one interactable serves all three input devices
Add gaze to an object that already responds to a controller
Identify which parts of a gaze interactor depend on the device
Configure the fallback that returns an object to ray control

Before you begin

You need the following in your scene:

The comprehensive interaction rig. See Create the Comprehensive Interaction Rig.
An object that already responds to one interaction, such as a ray grab.
Eye tracking enabled on your target device, since gaze reads eye pose data.

Why there is nothing to convert

The comprehensive rig prefab adds support for hands, controllers, and controller driven hands to your scene in one step. Each device gets its own set of interactors under the rig. The object you interact with does not know or care which one reached it.

That split is the reason no conversion exists. The device-specific wiring lives on the interactor side of the rig. The interactable side holds one component per interaction type, and every device shares it. A RayInteractable on your object answers the hand ray interactor and the controller ray interactor equally.

The ConcurrentHandsControllersExamples scene ships this behavior as a working sample. See ConcurrentHandsControllersExamples Scene.

Gaze follows the same rule with one difference. Gaze is a separate interaction type, so it adds a GazeInteractable next to the interactables you already have. Both interactables route their events into the same Grabbable on your object, so the object behaves consistently no matter how the user reached it.

Layer	What varies by device
Rig interactors	One interactor per device per interaction type
Object interactables	One per interaction type, shared by every device
`Grabbable` on the object	One, shared by every interactable

Diagram of one interactable serving every device. Hand, controller, and controller driven hand ray interactors on the rig all reach a single RayInteractable on the object, and the hand and controller gaze interactors added with gaze reach a single GazeInteractable. Both interactables send events to the same Grabbable.

Add gaze with the Building Block

The Gaze Interaction Building Block is the fastest path. It installs a gaze interactable on the selected object, adds the gaze interactors your rig needs, creates the eye gaze system if the scene has none, and turns on ray fallback. Choose the Grab Object variant for a 3D object or the UI Canvas variant for a Unity Canvas.

The block adds gaze interactors for hands only. To also reach controllers, use the Quick Action in the next section, or add a Gaze interactor to a tracked controller through GameObject > Interaction SDK > Add Interactor(s) To Controller.

For what the block puts in your scene and how to inspect it, see Add Gaze Interaction with Building Blocks.

Add gaze with a Quick Action

Use a Quick Action when you want to choose the device types yourself.

In the Hierarchy, select the object that already carries your controller interaction.
Select GameObject > Interaction SDK > Add Gaze Grab Interaction. For a Unity Canvas, select Add Gaze Interaction to Canvas instead.
In the wizard, set Add Required Interactor(s) to the device types you want to reach. The default is Hands. Select Controllers as well to cover both.
Confirm that Gaze Surface or Gaze Collider is populated. Gaze needs one of them for hit testing. If neither is set, the wizard reports an error and offers a Fix button that generates a collider.
Leave Enable Ray Fallback selected unless you have a reason to turn it off. The next section describes what it does.
Select Create.

The wizard adds a GazeInteractable as a child of your object and points it at your existing Grabbable. It then adds the gaze interactors your chosen device types need, and wires each one to the scene’s eye gaze system.

Hierarchy panel showing the ISDK_GazeInteraction child added under the Slingshot object, alongside the existing ISDK_RayGrabInteraction child.

To learn more about the wizards, see Quick Actions.

What a gaze interactor needs per device

A GazeInteractor asserts three references at startup, and two of the three depend on the input device. Swapping the selector alone is not enough to move a gaze interactor between devices.

Property	Varies by device	What it holds
Selector	Yes	`IndexPinchSelector` on the `HandGazeInteractor` prefab, so a pinch confirms the selection. `ControllerSelector` on the `ControllerGazeInteractor` prefab, set to the trigger button.
Pointer Transform	Yes	A transform whose pose delta offsets the generated pointer events. The hand prefab uses a blended pointing pose. The controller prefab uses the controller pointer pose.
Candidate Provider	No	The scene’s `GazeConecaster`, parented under the `EyeGaze` GameObject. Every gaze interactor in the scene shares this one instance.

The wizards and the Building Block set all three for you. If you build a gaze interactor from your own code instead, supply all three together:

// selector:        IndexPinchSelector for hands, ControllerSelector for controllers.
// pointerTransform: the transform whose pose delta offsets pointer events.
// conecaster:      the scene's shared GazeConecaster.
gazeInteractor.InjectAllGazeInteractor(selector, pointerTransform, conecaster);

Leaving any of the three unassigned is a setup error rather than a supported configuration. For the selector half of this, see Selectors and gaze.

Configure the gaze and ray fallback

Gaze depends on eye tracking data. When that data is not available, an object that only answers gaze becomes unreachable. The Enable Ray Fallback setting exists to prevent that, and it is the part of this setup that most affects your existing ray interaction.

With the setting on, the Quick Action does two things:

It adds a TagSet component to the ray interactable on your object, carrying the tag ISDK_Gaze_Fallback. If the object has no RayInteractable in its hierarchy, the wizard creates one first.
It adds an ActiveStateTagSetFilter to each ray interactor for your chosen device types. The filter observes the scene’s EyeGaze component and checks for the same tag.

ActiveStateTagSetFilter defaults to Exclude mode, which behaves as a deny list. While the observed active state reports active, the filter removes tagged objects from that ray interactor’s candidates. While it reports inactive, every object passes.

The Active State Tag Set Filter component in the Inspector, with Active State set to OVREyeGaze, Mode set to Exclude, and Tag set to ISDK_Gaze_Fallback.

The EyeGaze component reports active while its tracked eye data is valid. The combined result is a clean handoff:

Eye tracking data	Ray interactor	Gaze interactor
Valid	Filtered away from the tagged object	Reaches the object
Not valid	Reaches the tagged object	No candidate

Diagram of the ray fallback handoff. When EyeGaze reports valid eye data, the ActiveStateTagSetFilter is active in Exclude mode, so the gaze interactor reaches the object and the ray interactor is filtered away from it. When the data is not valid, the filter is inactive, so the ray interactor reaches the tagged object and the gaze interactor has no candidate.

Both interactables stay in the scene the whole time. Nothing is enabled or disabled, and no components are swapped. The filter changes which candidates the ray interactor is allowed to score.

ActiveStateTagSetFilter also has an Include mode, which inverts the rule so only tagged objects pass while the active state is active. That mode is a general purpose tool and is not part of the gaze fallback setup.

To learn about active states, see Active State.

Verify the result

Enter Play mode with controllers tracked. Your original interaction should behave as it did before.
Put the controllers down and let hand tracking take over. The same object should respond to your hands, through the interactors the rig already had.
Look at the object and pinch. The gaze interactor should take the selection.

If the object responds to gaze but no longer responds to a controller ray, check the ActiveStateTagSetFilter on the controller ray interactor. A filter in Include mode, or one pointed at the wrong active state, blocks the ray path.

Learn more

To learn how selectors supply the confirmation step for a gaze interactor, see Selector.
To learn how interactors find and score candidates, see Interactors.
To learn about the ray interaction that the fallback hands back to, see Ray Interactions.
To see hands and controllers used at the same time in a shipped scene, see ConcurrentHandsControllersExamples Scene.
To inspect everything the Gaze Interaction Building Block adds, see Add Gaze Interaction with Building Blocks.