Design

Choosing an interaction model

Updated: Sep 29, 2026

Direct vs. indirect interaction

Choosing an interaction model is one of the most important decisions you make early in design. It determines how players target, select, and manipulate content. It also creates consistent interaction logic across your experience.

Hands input supports two broad approaches:

Direct interaction: touching and manipulating objects in the virtual world.
Indirect interaction: using a hand ray, gaze with hand confirmation, microgestures, or other hand gestures.

Direct interaction strengthens embodiment when movement is part of the fantasy. Examples include holding items, tools, or weapons. Direct interaction also includes touching, grabbing, and manipulating the environment.

Indirect interaction works well when players need reach, precision, or comfort. For example, players aim at multiple distant targets in an immersive mixed reality experience.

Choose the model that works across most of your actions. This gives players a consistent way to interact.

🎓 See these trade-offs side by side: Tips for Choosing an Interaction Model⁠, a short video on Meta VR Academy, compares touch, ray, and gaze by task, distance, and precision.

Direct interaction	Direct interaction means touching and manipulating objects in the virtual world. Grab interaction using pinch and palm grab gestures to hold virtual objects.
Indirect interaction	Indirect interaction means using a hand ray, gaze with hand confirmation, microgestures, or other hand gestures. Ray interaction using a virtual ray extending from the user's hand to interact with distant objects.

Start simple

Start simple by building your design around one or two core interactions supported by Interaction SDK (ISDK).

Direct interaction	Touch Poke Long press Long press and drag Scroll Grab Pinch Palm Grab Use hand grab
Indirect interaction	Ray casting Eyes Microgestures

In Dragon Grove⁠, from Schell Games, players use a primary pinch interaction for all main gameplay.

This includes plant interactions.

A pinch interaction is used for Dragon Grove's plant gameplay.

Players also use pinch for physical manipulation of the environment. This includes positioning the board, opening drawers, and picking up tags. This consistent input helps players learn the interaction and recall it when they return to the game.

The same pinch interaction supports physical manipulation of the board, drawers, and tags.

Multiple interaction modes

Take care when supporting multiple interaction modes at the same time. For example, players can pick up board game pieces directly by pinching. They can also target a piece with a hand ray and pick it up indirectly with a pinch.

When supporting direct and ray interaction together, define custom input handling that resolves targeting priorities. Refining both interaction modes increases design and implementation complexity.

Choose the interaction model for menus intentionally

Choose the interaction model for each menu deliberately.

Match menus to your primary interaction model or choose a different model for the menu’s needs. For example, use direct-touch menus in a direct-touch title, or use ray casting for menus with denser information.

For more guidance, see Hands UI best practices.