Design

Hands Interaction Types

Updated: Oct 22, 2025

Interaction Types

The following is an introduction to the different types of hands interactions and best practices that can be used in your app with interaction SDK. For links and examples on implementation of the interaction types, see Next steps.

Ray

Ray interaction involves using a virtual ray, typically extending from the user’s hand, to interact with distant objects in a virtual environment. This method is ideal for selecting or manipulating objects that are out of reach.

Best practices for using hands input with ray interaction

Use visual affordances such as a cursor and/or object-specific hover states to indicate which object is currently targeted by the ray.

Use pinch gestures for hands to perform selections.

Implement visual and audio feedback to confirm successful selection.

User ISDK’s “first/best hover” option to prioritize in scenes with multiple interactables.

Don't

Make interactive elements too small or closely packed, as this can lead to accidental selections when the user is slightly moving while performing a pinch/selection.

Rely solely on ray interaction; consider combining it with other interactions for targeting or poke for direct interactions.

Distance Grab

Distance grab allows users to grab or manipulate objects from a distance. There are three methods of distance grabbing showcased in ISDK:

Interactable to Hand: Moves the item rapidly toward the hand, where it then stays attached for further interaction.

Anchor at Hand: Enables users to move the item from a greater distance while maintaining its position relative to the hand.

Hand to Interactable: Displays a copy of the hand at the object’s position, allowing for precise manipulation of the distant object.

Best practices for using hands input with the distance grab interaction

Use visual and audio cues to indicate which object is currently targeted for distance grabbing.

Implement magnetism to make targeting simpler and more satisfying.

Don't

Rely solely on distance grab for all interactions; consider combining it with direct grab for a more versatile user experience.

Poke

Poke refers to the interaction of touching a virtual button or panel with the index finger.

This gesture is used for direct interactions, such as pressing UI buttons or scrolling panels.

Touch allows for quick selection of buttons and UI, at the expense of some effort. Human experience and decades of touch based computing make this interaction incredibly intuitive, but Z distance needs to be clearly communicated for virtual interfaces to succeed. Direct touch interactions have constrained selection criteria and are typically hard to confuse for pinching. Though, it can coexist with Hands in a multimodal system.

Best practices for using hands input with the poke interaction

Use touch limiting to prevent the virtual hand from intersecting with the panel or virtual object, improving the UX.

Incorporate scrolling and long press functionalities where appropriate to expand interaction capabilities.

Use clear and consistent visual and audio feedback when an object is poked to confirm successful interaction.

Dissociate the virtual hand from the physical hand using limiting techniques.This helps simulate the confidence and feedback users expect from real-world haptic surfaces, making virtual interactions feel more reliable.

Encourage direct touch selection with the index finger. Design your UI so that selection occurs only when the fingertip is actually touching the interface. This mirrors natural touch behavior and avoids confusion with gestures like pinching.

Place direct touchable elements on natural, manipulatable surfaces. Position buttons and interactive elements where users would intuitively expect to reach and touch, inviting tactile engagement and making the experience more immersive.

Don't

Make interactive elements too small or closely packed, as this can lead to accidental pokes or missed interactions. Use a minimum target size of 22mm x 22mm with a minimum of 12mm spacing between interactables.

Rely solely on poke; consider combining it with indirect interactions like raycasting for targeting, and pinch for selection, to enhance user experience. Pinching is a different gesture and should not be used for direct touch interactions. Reserve direct touch for index finger contact only.

Enable poke for other fingers besides the index finger which suffer from lower accuracy due to self-occlusion when tracking.

Place touchable elements in unnatural or hard-to-reach locations. Avoid positioning UI elements where users wouldn’t naturally reach or where it feels awkward to interact, as this breaks immersion and usability.

Grab

Grab describes the interaction of holding on to a virtual object. The supported and recommended gestures to do so are:

Pinch Index and thumb touching.

Palm Grab Defined fingers curled or touching the hand palm.

Best practices for using hands input with the distance grab interaction

Use Hand Pose for grab to make it look/feel better. Use ISDK’s mirrored HGI option to replicate from one hand to the other.

Consider implementing different grab types and hand poses based on the object's size, shape, and “virtual weight”.

Test your implementation with various hand sizes and shapes to ensure inclusivity.

Use visual and audio feedback to make up for lack of haptics.

Don't

Not all fingers need to be curled at the same time to hold onto a virtual object.

Assume that users will always use the same hand or fingers to interact with objects.

Pose and Gesture Detection

Detect specific hand poses and gestures from users to trigger in-app events (Example: thumbs up to emote).

Pose: Static hand position (Example: thumbs up).

Gesture: Hand movement over multiple frames (Example: waving).

Best practices for using hands input with the pose and gesture interaction

Limit use to a small number of very simple poses and gestures; gesture learnability and memorability is a challenge.

Provide visual instructions when teaching users poses and gestures; verbal or written instructions are prone to varied interpretations.

Test pose and gesture-based mechanics with a broad set of test users to ensure detection reliability before releasing.

Consider adding gating mechanisms, such as proximity (gesture works only when close to a certain object) or hover state to prevent accidental activations.

Don't

Use gestures that may be culturally inappropriate.

Assume initial gesture designs will work without iterative testing and feedback. Continuously test your gestures for accidental activations.

Use poses or gestures that will be prone to self-occlusion (Example: palm occludes fingers from tracking)

Locomotion

Locomotion refers to the ability to move around in a virtual environment. In VR, different input modalities can be used for moving yourself or characters. For hand-based locomotion, there are two key design considerations:

Gating: how users enter and exit locomotion mode.

Interaction: how users interact while in locomotion mode.

Best practices for using hands input with the distance grab interaction

Use gating to avoid false positives and prevent conflicts with other interactive actions within the environment. Gating provides a clear transition between idle and locomotion modes.

Offer both snap and continuous locomotion styles to accommodate user preferences and comfort levels.

Consider that users might need to hold objects and move using their hands while in locomotion mode.

Don't

Assign turn and movement controls to different hands, the head, or the body, as requiring simultaneous actions from multiple areas can be confusing and inaccessible for minority of users.

Long Press Selection

Long-press is when a user holds down the selection of a button or object in order to show additional non critical information and functionality, leveraging the same expectations from existing mobile devices. The long press element should provide visual (button scale shown here) and auditory feedback when it has been pressed and a light ripple is used here to show the press emanating across the surface.

Best practices for using hands input with direct touch and long press interaction

Use long press to close interaction gaps with Android apps. Implement long-press functionality to provide familiar interactions for users transitioning from mobile platforms, especially when replicating secondary actions.

Trigger long press after a clear press delay. Set a press delay (Example: 0.5 seconds) before activating the long-press action, ensuring users have time to intentionally perform the gesture.

Treat early release as a short tap/press. If the user releases their finger before the press delay is met, register the action as a short tap or press, not a long press.

Maintain a click cooldown consistent with standard buttons. Use a click cooldown of 0.25 seconds for long-press buttons, matching the behavior of regular button interactions to prevent rapid, unintended activations.

Don't

Rely on long press as the primary interaction for VR-native experiences. Avoid using long press as the main method for critical actions in VR, as it is less discoverable and may not be as enjoyable as interactions designed specifically for immersive environments.

Activate long press without a clear delay. Ensure that long-press actions only trigger after the user has intentionally held their poke or pinch for the required duration, preventing accidental activations.