Hands Prototype Quickstart Guide

• Try hands-based apps — Download First Hand⁠ and the Interaction SDK Samples⁠ on your headset to see what polished hand interactions look and feel like. See Hands Examples for more titles to try.
• Review the design guidance — Read the Hands overview to understand how hand tracking works in Meta Horizon OS, and check Hands Interaction Types to learn when to use poke, ray, and grab.
• Follow the best practices — The Hands Best Practices guide covers ergonomics, common pitfalls, and do’s and don’ts for hand-based interactions.

---

Set up the camera and hand tracking

Add the Interaction SDK camera rig to your scene and enable hand tracking. This replaces the default Main Camera with a rig that supports hands, controllers, and common interactions out of the box.

01 Add the Interaction Rig

02 Configure the Wizard

03 Create the Rig

04 Enable Hand Tracking

Right-click in the Hierarchy and select Interaction SDK > Add OVR Interaction Rig.

The Comprehensive Rig Wizard opens. If the wizard says there's no OVRCameraRig in the scene, click Fix All, it will create one for you.

Click Create. The wizard adds the OVR Interaction Rig to your scene with everything wired up, hand tracking, controller tracking, raycast, poke, grab, and teleport interactions are all included.

OvrCameraRig features in the prefab.

1. Select OVRCameraRig in the Hierarchy.
2. In the Inspector, go to OVR Manager > Quest Features > General tab.
3. Set Hand Tracking Support to Controllers and Hands (recommended for prototyping). This lets you switch between hands and controllers freely.

Design your interactions so users can keep their arms close to their body with elbows at hip level. Requiring hands above heart level causes rapid fatigue. Place interactive content at a comfortable distance, about 1 to 1.5 meters away, for UI panels. See Hands Best Practices for ergonomic guidelines.

For any troubleshooting issues, see:

• Interaction SDK Getting Started
• OVR Interaction Rig Quick Action

---

Set up the panel

Create a world-space Canvas and add a Panel to serve as the background surface for your UI elements.

01 Create the Canvas

02 Configure the Canvas

03 Add a Panel

In the Hierarchy, right-click and select UI (Canvas) > Canvas. A new Canvas appears in the Hierarchy.

Right-click and select UI > Canvas.

1. Select the Canvas in the Hierarchy.
2. In the Inspector, set Canvas > Render Mode to World Space.
3. Set the Rect Transform values to PosX: 0, PosY: 1.5, PosZ: 1.5, Width: 480, Height: 720, and Scale: 0.0005 on all three axes (X, Y, Z).

This positions the Canvas at eye level about 1.5 meters in front of you and scales it down to a comfortable size for VR. For more approaches to building UI, including the UI Set toolkit, see ISDK UI Overview.

Right-click on the Canvas in the Hierarchy and select UI > Panel. The Panel gives you a background surface for your UI elements. You can change the panel color in the Inspector under Image > Color (check our color guidelines for more).

Avoid pure white (#FFFFFF) and pure black (#000000) in your UI. They cause eye strain in Passthrough or MR. Use a maximum background brightness of #DADADA for light themes. Colors appear more saturated in-headset than on your monitor, so always verify on the actual device. See Display Guidelines for more details.

---

Add components

Add interactive UI components to your panel. A button, a slider, and a toggle to give you a feel for the different UI patterns available for hand interaction.

For hand tracking, touch targets must be at least 48dp x 48dp (about 22mm x 22mm) and spaced at least 12mm apart to prevent accidental touches. The button size of 300 x 100 in this guide exceeds that minimum, but keep these numbers in mind as you add more elements. For text, use a minimum font size of 14px (18px recommended for comfortable reading) and maintain a 4.5:1 contrast ratio. See Hands Interaction Types and Fonts and Icons for detailed specs.

01 Add a Button

02 Add a Slider

03 Add a Toggle

04 Canvas Validation

1. Right-click on the Panel and select UI > Button - TextMeshPro.
2. In the Inspector, set the Rect TransformWidth to 300 and Height to 100 to make it a comfortable size for hand interaction.
3. You can change the button label text by expanding the button in the Hierarchy and selecting the text child object.

1. Right-click on the Panel and select UI > Slider.
2. Set the Width to 300 and Y Pos to be above or below the button.
3. Sliders work with both poke (drag the handle with your finger) and ray interactions. For design guidance on slider types, see slider design guidelines.

1. Right-click on the Panel and select UI > Toggle.
2. Set Width to 300 and Height to 80 and the Y Pos to be above or below the other UI components.
3. Toggles work well for on/off settings. Feel free to add more components, dropdowns, input fields, or additional buttons, to build out your prototype UI.

When done, your canvas should look something like this with all your components evenly spaced.

Validate your canvas is set up with all its components.

For production-quality UI, consider using the Meta Horizon OS UI Set prefabs instead of standard Unity UI. Find them in Project > Packages > Meta XR Interaction SDK Essentials > Runtime > Sample > Objects > UISet > Prefabs. These include pre-styled buttons, sliders, toggles, and more with built-in interaction support.

For UI guidelines, see:

• UI Set Components
• Slider Design Guidelines
• Components Design Guidelines

---

Add hand interactions to the UI

Now connect the Interaction SDK to your Canvas so you can poke buttons with your fingertips and use ray interactions from a distance.

Always support both poke and ray on UI surfaces. Don't rely on only one. Some users prefer direct touch, others prefer pointing from a distance, and some may have accessibility needs that favor one over the other. Also, hands have no haptic feedback unlike controllers, so provide clear audio and visual cues (color changes, animations, sounds) on every interaction. See Accessibility Guidelines for inclusive design practices.

01 Add Poke Interaction

02 Add Ray Interaction

03 Verify Setup

1. Right-click on the Canvas in the Hierarchy and select Interaction SDK > Add Poke Interaction to Canvas.
2. The Poke wizard appears. If it flags a missing PointableCanvasModule, click Fix. This adds the PointableCanvasModule to your EventSystem, which routes touch events to your Canvas.
3. Click Create. You can now poke buttons, drag sliders, and tap toggles with your fingertips.

1. Right-click on the Canvas again and select Interaction SDK > Add Ray Interaction to Canvas.
2. The Ray wizard appears, fix any missing components if prompted, then click Create.
3. This lets you interact with your UI from a distance using a ray that projects from your pinched fingers or controller.

Your Canvas should now have PokeInteractable, RayInteractable, and PointableCanvas components attached (you can verify in the Inspector). The Interaction SDK rig you added earlier already has the hand and controller interactors, so everything is wired up. You're ready to test!

Canvas with interaction components attached.

For any troubleshooting issues, see:

• Poke Canvas Quick Action
• Ray Canvas Quick Action
• Canvas Integration

The fastest way to iterate, no headset required. Meta XR Simulator runs your app directly in the Unity Editor using keyboard and mouse input to approximate head and hand tracking. Works on both Windows and macOS (Apple Silicon only, Unity OpenXR Plugin v1.13.0+, Meta XR SDK v66+).

Best for: Rapid prototyping, layout testing, verifying interactions work, macOS development.

Limitations: Input is approximated (not real hand tracking), rendering and performance won’t match the actual headset. You must still do a final on-device test before shipping.

01 Activate the Simulator

02 Press Play

03 Switch to Hand Input

04 Test Your UI

Click the XR Simulator icon next to the Play button in Unity's top toolbar, or go to Meta > Meta XR Simulator > Activate. You'll see a console log confirming [Meta XR Simulator is activated].

Activate Meta XR Simulator.

Check to see if Meta XR Simulator is activated.

Press the Play button in the Unity Editor. The Meta XR Simulator window opens, showing your scene from the simulated headset's perspective. Use the mouse to look around and WASD keys to move.

Seeing an empty world with no hands, controllers, or canvas? If you get the error No Render Pipeline Asset was found, your project is missing a Universal Render Pipeline (URP) asset. OpenXR requires URP to apply XR rendering fixes. To fix this:

1. If you don't have a URP Asset yet, right-click in the Project window and select Create > Rendering > URP Asset (with Universal Renderer).
2. Go to Edit > Project Settings > Graphics and assign your URP Asset in the Scriptable Render Pipeline Settings field.
3. Go to Edit > Project Settings > Quality and assign the same URP Asset for each quality level you use.

1. In the Meta XR Simulator panel, go to Inputs > Global input settings.
2. In the Left and Right dropdowns, switch from Controller to Hand.

1. Move your simulated hands toward the Canvas and try poking the button, dragging the slider, and tapping the toggle.
2. Keys 1 through 4 switch between aim, poke, pinch, and grab poses. Left mouse button triggers a pinch gesture.
3. When you're done, press Play again to stop.
4. Go to Meta > Meta XR Simulator > Deactivate before switching to headset testing.

For any troubleshooting issues, see:

• Meta XR Simulator Overview
• Getting Started with Meta XR Simulator

Meta XR Simulator supports additional features beyond what this guide covers, including controller emulation, room configuration, and synthetic environments. See the Meta XR Simulator overview to learn more.

• Hands Overview — How hand tracking works in Meta Horizon OS and how to think about designing for hands
• Hands Interaction Types — Deep dive into poke, ray, grab, and when to use each
• Hands Best Practices — Ergonomic guidelines, common pitfalls, and do’s and don’ts
• Hands Examples — Real app examples of hands-based interactions for inspiration
• Hand Representation — How to render hands in your app (system hands, custom models, transparency)
• Panels — Panel sizing, placement, and layout for spatial UI
• Accessibility — Inclusive design for diverse hand sizes, abilities, and input preferences

• Interaction SDK Overview — Architecture and concepts behind the SDK
• Quick Actions — All available right-click Quick Actions for rapid setup
• Hand Grab Interactions — Let users pick up and manipulate virtual objects
• Hand Pose Detection — Detect custom hand poses and gestures
• Custom Hand Models — Replace the default hand mesh with your own
• ISDK Example Scenes — Pre-built scenes demonstrating grabs, pokes, rays, and more
• Microgestures — Thumb-based gestures for shortcuts and controls