Develop

Getting Started with Interaction SDK

Updated: Sep 14, 2026
This tutorial explains how to create your first interactions using Interaction SDK. You will add the Interaction SDK rig to your scene, set up a UI, and add interactions to enable users to interact with the UI.
The Building Your First Game for Hands⁠ interactive course on Meta VR Academy walks you through the full journey, from choosing a core loop to testing on-device.
Runtime Unity XR and the Project Setup Tool
If you must use Unity XR instead of the Meta XR Core SDK, see Getting Started with Interaction SDK and Unity XR.

Prerequisites

Before proceeding with this tutorial, complete the setup steps outlined in the following sections:

Step 1: Add the rig

In Interaction SDK, the rig is a predefined collection of GameObjects that enables you to see your virtual environment and initiate actions, such as grabbing, teleporting, or poking. The OVRComprehensiveInteractionRig prefab contains this rig. It integrates many of the core interactions and features offered by Interaction SDK, wired up according to best practices, including support for poke, ray, multiple types of grabs, and locomotion. It also adds support for hands, controllers, and controller-driven hands to your scene.
The rig reads hand, controller, and headset data from an OVRCameraRig, which handles the camera system and head-movement tracking. Adding the rig as a child of the OVRCameraRig is what the Quick Action does, and it lets the editor resolve the rig’s camera reference by searching the parent objects. If you place the rig elsewhere in the Hierarchy panel, assign the camera rig on the rig’s OVRCameraRigRef component yourself.
Instead of manually adding these prefabs to the scene, we recommend using Interaction SDK Quick Actions, which finds or adds any missing dependencies such as the camera rig. You can read a more advanced guide about how to use this rig in Comprehensive Interaction Rig.
Gaze also works with the UnityXR rig. To use that rig instead, select Interaction SDK > Add UnityXR Comprehensive Interaction Rig and follow Create the UnityXR Interaction Rig. The steps below describe the OVR rig, as does the OVR Manager eye tracking step later on this page.
  1. Delete the default Main Camera if it exists, since Interaction SDK uses its own camera rig.
  2. Right click on the Hierarchy and select the Interaction SDK > Add OVR Comprehensive Interaction Rig Quick Action.
    Using the Interaction SDK Rig Wizard
  3. If you have an OVRCameraRig in the scene, it appears referenced in the wizard. If there is no camera rig, click Fix All so the wizard creates one.
    Fix All Button
  4. Select Generate As Editable Copy and set the Prefab Path to store an intermediary copy of the rig prefab so you can store as many overrides as needed.
    The OVR Comprehensive Interaction Rig Wizard with the Generate As Editable Copy checkbox selected and the Prefab Path set to InteractionSDK/ComprehensiveInteractors.prefab.
  5. If you want to further customize the rig, adjust the settings in the wizard. For details on the available options, see the OVR Comprehensive Interaction Rig Quick Action documentation.
  6. Click Create to add the OVR Comprehensive Interaction Rig to the scene.
    Hierarchy panel showing the OVRComprehensiveInteractionRig GameObject added as a child of OVRCameraRig.
  7. In the Hierarchy, select the OVRCameraRig.
  8. On the Inspector tab, go to OVR Manager > Quest Features, and then on the General tab, in the Hand Tracking Support list, select Controllers and Hands, Hands Only or Controllers only depending on your needs. The Hands Only option lets you use hands as the input modality without any controllers.
    The Quest Features General tab in the Inspector with Hand Tracking Support set to Controllers And Hands.
Note: The Hand Tracking Support step above assumes hand tracking, which is the input this tutorial uses throughout. Gaze runs with controllers too, so choose the option that matches the input devices your project ships. See Reach controllers as well as hands for the interactor setting that goes with that choice.

Step 2: Set up your UI

  1. In the Project panel, navigate to the Packages > Meta XR Interaction SDK Essentials > Runtime > Sample > Objects > UISet > Prefabs > Backplate folder and add a backplate for the UI by dragging the EmptyUIBackplateWithCanvas prefab to the Hierarchy panel.
    Drag and drop the backplate prefab
    The backplate prefab contains a Canvas, a background for the UI, some basic layout components, and ray and poke interactable components to enable direct touch and raycast interactions with the UI.
    Scene view showing the EmptyUIBackplateWithCanvas prefab placed in the scene.
  2. In the Hierarchy, select the CanvasRoot. In the Inspector, under Rect Transform, use the Width and Height properties to set the size of the Canvas. This example uses the following settings to scale it to a reasonable size for a few components:
    • Rect Transform > Width: 500
    • Rect Transform > Height: 250
    The CanvasRoot Rect Transform in the Inspector with Width set to 500 and Height set to 250.
  3. In the Hierarchy, select the UIBackplate. In the Inspector, under Rect Transform, set the Width and Height properties to match the Canvas width and height set in the previous step. This example uses the following settings to match the Canvas size:
    • Rect Transform > Width: 500
    • Rect Transform > Height: 250
    The UIBackplate Rect Transform in the Inspector with Width set to 500 and Height set to 250.
  4. Add some UI elements to the panel to interact with by dragging and dropping prefabs from the Packages > Meta XR Interaction SDK Essentials > Runtime > Sample > Objects > UISet > Prefabs folder. For example, in the Buttons > UnityUIButtonBased folder, drag the PrimaryButton_IconAndLabel_UnityUIButton prefab to the UIBackplate object in the Hierarchy. The button element appears on the UI. Add as many elements as you desire to create your UI.
    UI with elements

Use eye tracking to target elements

Gaze targeting reads eye pose data, so your project must declare eye tracking support before it runs on a device. In the Hierarchy panel, select the OVRCameraRig. In the Inspector panel, under OVR Manager > Quest Features > General, set Eye Tracking Support to Supported or Required. Eye Tracking Support is a dropdown rather than a checkbox. Choose Supported if your app still runs without eye tracking, or Required if it does not.
The Quest Features General tab in the Inspector with Eye Tracking Support set to Supported.
  1. Right-click on the Canvas, and select Interaction SDK > Add Gaze Interaction to Canvas. The wizard for that menu item appears.
    The quick actions menu.
  2. In the wizard, you may see a message about a missing PointableCanvasModule. Select Fix. This adds the module to the scene’s EventSystem object, or creates an object that carries both. PointableCanvasModule is a scene-level singleton, so it does not go on the Canvas.
    The fix all option
  3. If you want to further customize the interaction, adjust the interaction’s settings in the wizard. For details on the available options, see the Gaze Canvas Quick Action documentation.
  4. Select Create. The wizard automatically adds the required components for the interaction to the Canvas. It also adds components to the camera rig if those components weren’t already there.
    The automatically added GameObjects for a gaze-able UI.

Step 3: Make the UI grabbable

To make the UI grabbable, a best practice is to add a handle to the UI for the user to grab onto instead of grabbing the UI panel itself. This helps to avoid conflicts between grabbing the UI and interacting with the UI elements. The handle is a sibling child object to the UI under a common parent, so when you add the grab interaction to the handle, you can set the target transform of the Grabbable to point to the transform of the root object and move the UI and handle all at once.
Diagram of the grab handle hierarchy. UIContainer is an empty GameObject with two children, the UI backplate and the GrabBar handle. The GrabBar carries the Image and the gaze grab interaction, whose Target Transform points back at UIContainer.

Add a grab handle

  1. Right-click in the Hierarchy panel and select Create Empty to add a new GameObject. This acts as a container for the UI and handle. Name the object accordingly. In the example, it is called UIContainer.
  2. In the Hierarchy panel, drag your UI onto the new UI container object to parent it.
    Hierarchy panel showing the UI object parented under the UIContainer GameObject.
  3. In the Hierarchy panel, right-click on the UI container object and choose Create > UI > Canvas. Name the new object GrabBar. In this example, the handle is composed of a Canvas with an Image that has a rounded box material applied. You can use a different visual for your handle as long as it has a surface to interact with. In the Canvas component properties, set the Render Mode to World Space and Enable TexCoord1 in the Additional Shader Channels. Set the Scale, Width, and Height for the Rect Transform component of the GrabBar object to set the size appropriately for your use case. In this example, they are set to:
    • Width: 300
    • Height: 20
    • Scale: (0.0005, 0.0005, 0.0005)
    Canvas Rect Transform in Inspector with Width 300, Height 20, and Scale 0.0005.
  4. Place the handle object directly beneath the UI you want to grab.
    Scene view showing the GrabBar handle positioned directly beneath the UI panel.
  5. In the Hierarchy panel, right-click on the GrabBar object and choose Create > UI > Image. Set the Width and Height for the Rect Transform component of the Image to match the dimensions set on the Canvas previously. In this example, they are set to:
    • Width: 300
    • Height: 20
    Inspector showing the Image Rect Transform with Width 300 and Height 20 matching the Canvas.
  6. In the Project panel, click on Packages > Meta XR Interaction SDK Essentials > Runtime > Sample > Materials > RoundedBoxUI.mat and drag it onto the Material property of the Image object in the Inspector panel.
    Dragging RoundedBoxUI material from Project panel onto the Image Material property in Inspector.
  7. In the Inspector panel with the Image object selected, click Add Component and select Rounded Box UI Properties. Set the Border Radius property to:
    • X: 20
    • Y: 20
    • Z: 20
    • W: 20
    The Image is displayed with rounded corners.
    GrabBar Image displayed with rounded corners after setting Border Radius to 20 on all axes.
  8. The default Movement Provider added by the Gaze Grab quick action is Move From Target Provider. For moving UIs, we recommend Coronary Movement Provider instead, because it can billboard the UI to face the user while it is grabbed. In the Inspector panel with the GrabBar object selected, click Add Component and select Coronary Movement Provider. You assign it in the wizard later.
    Note: CoronaryMovementProvider is experimental. Experimental types can change without a deprecation cycle.
    The Coronary Movement Provider component in the Inspector, showing the Hmd, Occipital Offset, Coronary Offset, Depth Remapping Factor, Billboard To Face User, and Billboard Pitch Offset fields.

Add the gaze grab interaction

  1. Right-click on the GrabBar object and select Interaction SDK > Add Gaze Grab Interaction. The wizard for that menu item appears.
    The Gaze Grab Wizard
  2. In the wizard, select the Fix and Fix All buttons to fix any errors. This adds missing components or fields where they are required.
    The fix all option
  3. In the wizard, set the Target Transform to point to your UI container object. This causes the grab interaction to affect the entire UI and not only the handle object.
    Gaze Grab wizard with Target Transform set to the UIContainer object.
  4. In the wizard, set the Movement Provider to the Coronary Movement Provider on the GrabBar object that you added previously. The grab interaction then uses that provider to move the UI.
    The Optional Components section of the Gaze Grab wizard with Movement Provider set to GrabBar (Coronary Movement Provider).
  5. If you want to further customize the interaction, adjust the interaction’s settings in the wizard. For details on the available options, see the Gaze Grab Quick Action documentation.
  6. Select Create. The wizard automatically adds the required components for the interaction to the GameObject. It also adds components to the camera rig if those components weren’t already there.
    The automatically added GameObjects for a gaze grab interaction.

Reach controllers as well as hands

Both quick actions in this tutorial expose an Add Required Interactor(s) setting that controls which devices receive a gaze interactor. It defaults to Hands, so following the steps as written gives you gaze on hand tracking alone. To reach controllers too, select Controllers in that field before you select Create. The setting accepts more than one device, and the rig runs the resulting interactors side by side rather than replacing one with the other.
For the full picture of serving hands, controllers, and gaze from one object, see Support Hands, Controllers, and Gaze on the Same Object.

Keep the UI reachable without eye tracking

Gaze depends on eye pose data. Without it, an object that answers only gaze cannot be reached at all.
Both quick actions in this tutorial leave Enable Ray Fallback selected by default, so the UI you built already handles this. While eye tracking data is valid, ray interactors are held off the UI and gaze targets it. While that data is not valid, the ray interactors reach the UI instead. If the object you run the quick action on has no ray interactable, the quick action creates one so that ray has something to fall back to. Clear Enable Ray Fallback only if you have a reason to leave the object reachable by gaze alone.
Diagram of the ray fallback decision. With Enable Ray Fallback selected, valid eye tracking data gives the UI to the gaze interactor and holds ray interactors off, and invalid data lets ray interactors reach the UI through its RayInteractable. Clearing the setting leaves the UI unreachable without eye tracking.
For the components this setting adds and how the handoff is filtered, see Support Hands, Controllers, and Gaze on the Same Object.
Use Link to test your project.
  1. Open the Link desktop application on your computer.
  2. Put on your headset, and, when prompted, enable Link.
  3. On your development machine, in Unity Editor, select the Play button.
  4. In your headset, you can interact with the UI directly or at a distance using ray-casting. The UI can be moved around by grabbing it.

Test your interaction by generating an APK

Build your project into an .apk to test your project.
  1. Make sure your headset is connected to your development machine.
  2. In Unity Editor, select File > Build Profiles.
  3. Click Open Scene List to open the Scene List window.
  4. Add your scene to the Scene List by dragging it from the Project panel or by clicking Add Open Scenes.
  5. Click Build and Run to generate an .apk and run it on your headset. In the File Explorer that opens, select a location to save the .apk to and give it a name. The build process may take a few minutes.
  6. In your headset, you can interact with the UI directly or at a distance using ray-casting. The UI can be moved around by grabbing it.

Learn more

Next steps

Add some GameObjects and make them interactable with Quick Actions.
Meta
English (US)
© 2026 Meta