General information for immersive experiences
Passthrough vs. natural vision
Passthrough converts the 3D physical world into a 2D representation.
- Depth perception may be reduced compared to natural vision.
- Visual distortions can occur, leading to inaccurate representations of object shape, size, and distance.
- Grayscale passthrough (Meta Quest 2 and prior) is lower in quality than color passthrough (Meta Quest Pro and beyond).
- Higher display resolution and color aid in better depth perception, visual acuity, and object recognition.
- Colored virtual content on a grayscale background attracts attention and may distract from the physical environment.
Figure 1: Colored virtual content overlaid on grayscale passthrough in Meta Quest 2. Note the higher resolution and prominence of virtual objects compared to the physical environment.
Limited field of view (FoV)
Head mounted displays (HMDs) limit the FoV compared to natural vision.
- Limited FoV makes objects and bystanders harder to detect in the periphery, similar to wearing a ski mask.
- Users may not understand that their FoV is reduced and behave as usual.
- Most unintentional contacts occur for objects outside the FoV.
- FoV limitations vary across HMDs due to different form factors.
- Design choices should consider the most limiting HMDs with reduced FoV by more than half.
Figure 2: Natural vision (far left) compared to FoV limitations in an example HMD in immersive experiences (right images). Blur in the periphery illustrates natural visual acuity when fixating on a point in the scene. Blue represents the range of vision; red indicates portions of the visual scene blocked when donning the HMD.
Seated and stationary immersive experiences present less of a user challenge; however, many use cases allow users to make use of their full room and move around a large area.
- Fast user motion presents a challenge to users and movement types can be highly variable (e.g., reaching, dodging, walking, running).
- Limited FoV increases the likelihood that a user will not detect potential obstacles, especially at high speeds.
- Larger head movements are necessary for the user to see their feet while walking or notice intrusions in their activity space.
- This can distract the user from the task of navigating, especially when they are immersed in an experience.
- Approaching or looming content can lead to reactive behavior such as dodging quickly or backing up into a physical object.
- Removing the boundary removes the visual grid near borders of the activity space and does not alert users of potential obstacles outside the FoV.
- Vection (illusory self-movement) can arise when a user is stationary but content is moving and impact user comfort (e.g., motion sickness).
- For a physical world instance, sitting parked next to a car that is backing out.
- If the app is a high activity experience that promotes movement around the activity space, encourage the user to clear the area of potential obstacles, remain alert, and move slowly with intentional goals.
- Reducing opacity virtual content, dampening audio, or other audiovisual cues can assist users when:
- Moving at high speeds or in reverse.
- Getting close to physical objects, room boundaries, or virtual content not intended for close interaction.
Visual vs. physical affordance
In mixed reality, some virtual assets encourage interaction while others are intended to be avoided. Portals and windows allow users to peer into virtual worlds beyond their physical space, but accurate depth perception can be challenging when they are large and occupy most or all of the user’s FoV. Depth information can also lead to a sensation of dropping when users walk over portals placed on the floor.
Affordance refers to the actions a user perceives as possible for an object. In mixed reality, some virtual assets encourage interaction while others are intended to be avoided.
- The appeal of mixed reality is to blend the physical world with virtual content. Some experiences place portals or windows on room boundaries (walls, floors, ceiling).
- Portals and windows allow users to peer into virtual worlds beyond their physical space, creating the illusion of a space that can be entered (see also Occlusions with virtual content).
- Despite design intentions, users may try to look or reach into portals or windows anchored to room boundaries.
- Accurate depth perception can be challenging when content occupies most of the FoV.
- Depth information can also lead to a sensation of dropping when users walk over portals placed on the floor.
- Floor content also blocks the user’s feet while navigating.
Figure 3: Left: Users may have difficulty judging distance when peering or reaching into wall portals. Right: Walking over floor portals can block the user’s feet and cause a sensation of dropping. Enabling
Depth API can assist navigation for content placed on the floor, but does not resolve this user challenge.
To mitigate these issues, user education should inform users of intended interactions with portals on room boundaries and encourage them to avoid getting too close. Other solutions include:
- Reducing opacity or adding audio cues when users get close to wall portals.
- Creating a buffer zone to keep portals away from room boundaries.
- Using game design affordances to discourage interaction with certain portals (e.g., visuals, losing points, etc).
- Avoid placing large virtual content on the floor that blocks actionable space.
- Enabling Depth API allows users to see their feet when navigating through virtual content on the floor.
Mixed reality and fully imersive experience transitions for hybrid apps
Mixed reality and fully immersive experiences require time for users to adapt. When entering fully immersive experiences from passthrough, users can better predict their location in the activity space; however, they quickly lose their mental map and need to reorient when returning to passthrough.
- Some use cases allow users to choose between mixed reality or fully immersive experiences, while others use portals for mixed reality to fully immersive experience transitions.
- A boundary is recommended at all times for fully immersive experiences – portals intended for transitions should always spawn within it to keep users within the predefined activity space and reduce user friction.
Figure 4: Left: Portal placement is within the boundary, ensuring users are in a clear activity space when entering fully immersive experiences. Right: Portal placement is outside the boundary and in a non predefined play area while immersed.
User challenges can be mitigated by making transitions intuitive, providing user education and time to readapt, and allowing users control over their current state.
- Clearly educate users on whether portals are for transitions or to be avoided (see Visual vs. physical affordance).
- Visual information expands from the point of fixation when moving towards objects, and contracts when moving away (optic flow). Similar transition designs are consistent with user mental models of entering and exiting natural spaces.
- When transitioning between mixed and virtual realities, provide time for users to reorient and adapt before introducing task demands.
- Allowing users to choose between states (e.g., game settings, quick actions) increases comfort and provides intentional control.
For more health and safety guidelines, see the following: