Design

Spatial UI inside interactive 3D scenes

Updated: May 18, 2026

When spatial UI sits inside or above an interactive 3D scene, hand interaction creates problems that pure spatial UI does not. This is common in tabletop games, immersive simulations, and any app that mixes spatial content with menus. This page covers the three most common issues.

For pure spatial UI guidance (sizing, density, distance, interaction-type selection), see Hands UI best practices. For broader guidance on hands as an input modality, see Hands best practices.

Role confusion: Spatial UI controls inside a 3D world

When indirect controls (raycast or pinch) on spatial UI sit alongside 3D direct manipulation (grabbing, tapping objects), users can experience role confusion where it is not visually obvious which actions need direct physical interaction and which need indirect menu input. Put simply, users could reach for objects that need a menu click. They raycast at objects they could grab.

Design accordingly:

Make the input mode visually obvious for each target. Use distinct affordances for grabbable 3D objects and clickable spatial UI: different highlight styles, different cursor behavior, and different hover feedback.

Group by interaction type when possible. Cluster all the menu-driven actions on a clearly-bounded panel, and leave the direct-manipulation surface visually clean. Mixing the two in the same visual space is the worst case.

Use audio and visual feedback to teach the mode on first interaction. Distinct sounds for “you grabbed something” and “you selected a menu item” help users build the mental model quickly.

The middle distance problem

Transitioning between direct touch and ray casting fails when a panel sits at an awkward middle distance. The panel is close enough that the highlight or hover state appears, but too far for the user to physically reach it. Without a clear visual indicator, users do not realize they need to switch from reaching to raycasting. They reach for the panel, miss, reach again, and give up, never realizing the panel was in raycast range the whole time.

Design accordingly:

Avoid placing UI in the middle distance (roughly 0.5m to 0.8m from the user) when possible. Either bring the panel into clear direct-touch range (under ~46cm) or push it well into raycast range (1m or more).

Show a mode indicator when the panel transitions. When the panel exits direct-touch range, the cursor or hover style should change visibly to a raycast reticle so the user understands what changed.

Test the handoff zone explicitly. Mid-distance handoffs are where most users lose the thread. Verify the visual transition is unmistakable.

For more on how the system selects between targeting modes, see Input hierarchy and fallback behavior.

Constrain panel rotation

Panels should always orient toward the user so they can be easily read and interacted with. If you let users reposition panels, the panel’s pitch and yaw should update automatically so it continues facing the user. Hold the panel’s roll constant. Free-rotation modes can exist, but they should be opt-in (a deliberate gesture or modifier), not the default behavior of release-after-grab.

Dos and don’ts

DO Use visually distinct affordances for grabbable 3D objects and clickable spatial UI in the same scene. Different highlights, cursors, and hover states teach the user which input applies.

DON'T Use the same hover or selection feedback for 3D objects and spatial UI controls. When everything looks alike, users cannot tell which inputs apply where.

DO Show a visible mode change when a panel transitions between direct-touch and raycast range. The cursor or hover style should make the new mode unmistakable.

DON'T Place UI panels at middle distance (0.5 to 0.8m) without a clear mode indicator. Users will reach for them, miss, and not realize the panel is actually in raycast range.

DO Keep panels oriented toward the user. When the user repositions a panel, auto-update pitch and yaw so it stays facing them and hold roll constant.

DON'T Leave panels at free-floating, crooked angles by default after a grab. Users have to perform tedious extra gestures to straighten them.