Design Guidelines: Providing a comfortable hand tracking experience is essential for creating immersive and enjoyable apps. Refer to the Design Guidelines at the bottom of the page to learn about best practices and to minimize risks of user discomfort.

What are Hand Grab Interactions?

A Hand Grab interaction provides a physics-less means of grabbing objects that’s specifically designed for hands and controller driven hands. For the controller version of grab, see Grab Interactions. Unlike a Grab interaction, a Hand Grab interaction uses per-finger information to inform when a selection should begin or end using HandGrabAPI.

In addition, the Hand Grab interaction enables hands to conform to a set of pre-authored poses during a selection. To learn about best practices when designing for hands, see Designing for Hands.

By the end of this guide, you should be able to:

Explain what a hand grab interaction is

Define the essential properties and functionality of the HandGrabInteractor component.

Define the essential properties and functionality of the HandGrabInteractable component.

Explain how to specify what types of grabs the interaction supports.

Describe how to configure the conditions that begin and end a hand grab interaction.

Note: If you are just getting started with this Meta XR feature, we recommend that you use Building Blocks, a Unity extension for Meta XR SDKs, to quickly add features to your project.

How do Hand Grab Interactions Work?

The Hand Grab interaction has two components, Hand Grab Interactor, which is attached to each hand via the HandGrabInteractor prefab, and Hand Grab Interactable, which is attached to each grabbable object. These two components communicate to determine whether a Hand Grab interaction has taken place and how the interaction should behave.

Hand Grab Interactor

The Hand Grab Interactor component is attached to your hands via the HandGrabInteractor prefab. HandGrabInteractor searches for the best HandGrabInteractable candidate and visually aligns the synthetic hand with the interactable by using the Hand Alignment property of HandGrabInteractable and implementing IHandGrabState.

Some key components, such as the HandGrabAPI and SupportedGrabTypes, are discussed in the following sections, but the optional GripPoints and PinchPoints should be noted here:

The GripPoint specifies an offset from the Wrist that can be used not only to search for the best HandGrabInteractable available, but also as a palm grab without a HandPose, and as anchor for attaching the object.

The PinchPoint specifies a moving point at the center of the tips of the currently pinching fingers. It is used to align interactables that don’t have a HandPose to the center of the pinch.

When the target interactables have a HandPose, the Wrist is used for searching and aligning the item.

It is also possible to enforce a grab or release (selection/unselection) of a HandGrabInteractable by calling the method ForceSelect/ForceRelease.

HandGrabInteractable

The HandGrabInteractable indicates not only whether an object can be grabbed, but how it can move with the interactor, which fingers should start and end the grab, and how the hand should align to the object.

The HandGrabInteractable can also specify HandGrabPoses,defining the handedness, visual hand pose, visual finger constraints, and surface to which a hand can align. If no HandGrabPose is provided, the grab will be a pose-less grab anchored at the interactable pose.

A grabbable object can be composed of multiple HandGrabInteractables.

Scoring Modifier

At the HandGrabInteractable, use the Scoring Modifier parameter to specify the strategy for finding the best grabbing point.

Position Rotation Weight: When using GrabSurfaces, this parameter indicates whether target poses that are closer to the actual hand should score better (no matter the rotation), or if poses with a rotation most similar to the rotation of the user’s hand should score better.

Note

For small to mid-sized objects, poses with a rotation most similar to the rotation of the user’s hand should score better, since it reduces the wrist motion.

Supported Grab Type

Supported grab type determines which grab types the object responds to. Possible values are None, Everything, Pinch, Palm, and All.

Grab Rules

Grab rules determine which fingers can start and end grabs. There are two sets of rules, Pinch Grab Rules for pinch grabs, and Palm Grab Rules for palm grabs. These rulesets apply so long as the Supported Grab Type property includes them. Both rulesets have the following properties.

Property	Description
Thumb, Index, Middle, Ring, Max (pinky)	Determines if the finger is needed to trigger a grab. Can be Required, Optional, or Ignored. All fingers marked as Required must be performing the gesture in order to trigger a grab. If no finger is marked as Required, at least one finger marked as Optional must be performing the gesture to trigger the grab. Fingers marked as Ignored are completely ignored.
Unselect Mode	Determines when the grab ends. Can be All Released or Any Released. In All Released mode, the grab ends when all required and optional fingers are released. In Any Released mode, the grab ends when at least one of the required fingers is released. If there are no required fingers, the grab ends when at least one of the optional fingers is released.

Pinch Grab Rules Example

The key in the HandGrabExamples scene can be pinch grabbed. All fingers are marked as Ignored except for Thumb and Index, which are set to Optional, so your thumb and index finger can trigger the grab. Unselect Mode is set to Any Released, so the grab will end when you release at least your thumb or index finger.

Pinch grabbing a key before releasing it

Triggering the pinch grab with the thumb and index finger, and then ending the grab by releasing the index finger.

Palm Grab Rules Example

The torch in the HandGrabExamples scene can be palm grabbed. All the fingers are set to Optional, so you can trigger the grab using at least one finger or your thumb. Unselect Mode is set to All Released, so the grab will end when you release all of the fingers you used to trigger the grab.

Palm grabbing a torch before releasing it

Triggering the palm grab with just one finger or thumb, triggering the palm grab with all digits, and then ending the grab by releasing all of the digits.

Interactable Movements

By default, when grabbing the interactable, it will align itself with the interactor and then move 1:1 with it each frame. To customize this behavior, attach an optional IMovementProvider component to the Interactable using the Add Component button. There are multiple components to choose from:

MoveTowardsTargetProvider: This is the default behavior, but adding the component manually lets you adjust the alignment motion curve and timings:

Travel Speed: The speed in m/s at which the object will move.

Use Fixed Travel Time: Instead of speed in m/s, set the TravelSpeed to a fixed time in seconds.

Travel Curve: The animation curve for how the speed will be applied to the object. By default, this applies some easing.

MoveFromTargetProvider: Does not attract the object toward the interactor. Instead it anchors it at the interactor pose. Useful for constrained interactables.

FollowTargetProvider: Continuously moves the interactor toward the interactable with some damping.

Hand Alignment

It is also possible to indicate how the visual hand will move in relation to the interactable and when a HandPose will be applied.

AlignOnGrab: This behavior will automatically snap the hand visually to the interactable when a grab starts. This is the default behavior.

AttractOnHover: This will progressively apply the final HandPose as the strength of a hover intensifies.

Align Fingers On Hover: This will progressively apply the final poses for the fingers (not the wrist) as the user starts to close the hand during a hover.

None: No hand pose is applied to the visual hand.

HandGrabInteractable can also specify which fingers are required to start and end a grab. For example, a grab can be triggered when the middle or the index finger is pinching with the thumb, or when the ring and middle fingers are curled into a Palm grab. Releasing might require any or all of these fingers to release.

HandGrabPose

HandGrabPoses are the visual workforce behind HandGrab interactions. They provide the position and visual parameters for the hand.

The HandGrabPose specifies a pose relative to the object at which the wrist should anchor, additionally it can also complement this information with the following optional properties:

HandPose: This option indicates that this HandGrabPose expects the hand to adopt a particular visual pose, and whether the fingers should be visually locked, constrained, or can move freely.

Surface: This option defines a IGrabSurface along which snapping by this HandGrabPose can occur at any point, rather than just using the relative position of this pose for snapping.

When several HandGrabPoses are specified in a HandGrabInteractable, the Interactable will interpolate between them based on the LocalScale and the Scale of the hand. With this approach, it is possible to generate HandGrabPoses with slightly different HandPoses at slightly different scales (such as 1x, 0.8x, and 1.2x), so that the hands of the users visually align properly with objects. Alternatively you can also choose to keep the scale of the users’ hands at a Fixed value using a FixedScaleHand.

An object can have several HandGrabInteractables with each indicating the different poses by which it can be grabbed via HandGrabPoses. To learn more about this workflow, check the cup in the HandGrabExamples sample scene.

We recommend that you always use HandGrabInteractables on objects with a scale of 1. If a different scale is required, scale a nested gameObject with the Mesh and Colliders but keep the root Rigidbody at scale 1.

HandGrabAPI

The HandGrabAPI is the core used to detect a Hand Grab Select or Unselect. It indicates when an IHand started and stopped grabbing, as well as the strength of the grabbing pose. To operate, the HandGrabAPI uses two IFingerAPIs implementations. By default, FingerPinchGrabAPI is used for pinching grabs, and FingerPalmGrabAPI is used for palm grabs. However, you can inject other APIs instead, such as the FingerRawPinchAPI, which is useful when using controller driven hands.

Each HandGrabInteractor can specify the GrabTypes it will support. These can be Pinch, Palm, or both. Additionally, the HandGrabInteractable can specify the GrabTypes that it requires to perform a Grab (note that a Pinch-only interactable can not be grabbed by a Palm-only Interactor). These are used in conjunction with the Grabbing Rules in the interactable.

Use with controller driven hands

It is possible to use the HandGrabAPI as-is when using Controllers-as-hands, the curl values being measured will be read directly from the animated hand. But it is recommended to use a FingerRawPinchInjector component alongside the HandGrabAPI, this will force it to read the values directly from the controller-fed Pinch values instead, resulting in a more reliable interaction.

Hand Grab Posing

To enable visual Hand Grab Posing with Hand Grab Interactions, two additional components are required:

A SyntheticHand overrides the wrist and joint hand data based on the hand grab interaction.

A HandGrabStateVisual component constrains finger joint rotations so they look as if they are perfectly wrapped around an object grabbed during a HandGrab interaction. This component can also override the wrist pose of the hand that’s attracted to any HandGrabInteractable that is marked with the Hand Alignment AttractOnHover.

An example of Hand Grab Posing can be seen in the HandGrabExamples scene. In this scene, there are separate SyntheticHands for Hands and controller driven hands. You can avoid this duplication and use one Hand Visual for both input device types if desired.