This page examines head-based input methods for immersive experiences. It highlights the benefits and challenges of this approach, details its functionality, and offers ergonomic tips, design principles, and best practices.

The head is fundamental to human anatomy and plays a crucial role in our ability to exist and interact with the world around us.

In immersive experiences, the head serves the same purpose, enabling presence by determining the user’s orientation and position.

These are the different parts, characteristics and frequently used terms to be familiar with:

This section explores inside-out tracking. Its operation, with a particular emphasis on precision, calibration, and the inherent technological limitations.

Meta Quest headsets (HMD) use inside-out tracking, a technology that tracks the HMD’s position in 3D space using embedded sensors and cameras. These sensors include accelerometers, gyroscopes, and magnetometers, which work together to provide information about the HMD’s movement and orientation. This eliminates the need for placing external sensors in the environment, making the device more convenient and user-friendly.

Depending on the hardware version, some headsets may also include additional hardware features that provide even more ways for users to interact. These features can include an MR button placed on the side of the HMD that toggles passthrough, allowing users to easily switch between seeing their physical or virtual environment, or eye tracking sensors located inside the HMD that enable users to target objects by gazing at them.

Various factors can influence the accuracy of the headset’s tracking, including environment variability, lighting conditions, and more. For further details on these challenges and the strategies used to mitigate them, refer to the section on limitations and mitigations. Understanding these factors is crucial for designing effective and immersive user experiences.

Head Calibration involves the device recognizing the head’s position and orientation by measuring specific points in the user’s environment. This measurement is crucial for accurately determining the user’s viewpoint and aligning the virtual content accordingly. To enhance the quality of head tracking across various conditions, the model is trained with a diverse dataset, including different environmental setups, lighting conditions, user movements, and orientations.

Calibration is the process by which the device determines specific points in the user’s physical environment to accurately track movement and measure distances relative to these points. This process allows the device to pinpoint the user’s spatial position and orientation, ensuring the precise integration of virtual content within the user’s physical environment.

The device can recall previously set up environments, and new ones need to be configured using the boundary setup.

This section explores inherent limitations and challenges and offers some mitigation strategies via design or code.

The headset’s sensors have a certain range and angle from which they can detect the physical space around them, known as the tracking volume. This volume varies depending on the arrangement of sensors and cameras on the HMD. As a result, the direction in which the head is turned directly affects the moment at which tracked content, like hands, environment reference points, or objects, moves outside the tracked area, which can influence interaction.

Mitigation - Design: To ensure that all relevant user motions are accurately tracked, it is recommended you design interactions that keep the user’s attention (head direction) towards the requested interaction. For example, when using hand tracking to type on a virtual keyboard, place the text entry field close to the keyboard so that the user’s hands remain within the tracked area while they look up to read their text. This approach helps minimize the impact of tracking volume limitations on user interaction.

Occlusion refers to the blocking of sensor visibility by obstacles, which can hinder accurate tracking. Occlusions can arise from both the surroundings and the user’s body, referred to as Self-Occlusions. Scenarios may include clothing, hands or physical surfaces being too close to the HMD, which can limit the device’s tracking capability.

Mitigation - Design: Design interactions that avoid prompting the user to bring their hands close to their head or approach physical surfaces too closely, ensuring optimal tracking capability.

The quality and intensity of light can significantly affect the system’s ability to accurately detect the physical environment. For instance, being outside in bright sunlight can overwhelm the device’s sensors, which can result in virtual content freezing, headlock, or misplaced objects. Reflective or transparent surfaces can also pose challenges for accurate tracking.

Mitigation - Design: Our devices adapt to varying light conditions, with continuous improvements through updates and new models. To ensure optimal functionality, it is recommended you advise users to be mindful of their environment’s lighting, for example through a note at the beginning of an application.

This section covers utilization of the head for interaction. This includes input primitives, design principles, ergonomic factors to consider, and the key “do’s and don’ts”.

Discover various input capabilities and interaction methods utilizing the HMD as input modality:

For a comprehensive overview of all input modalities and their corresponding Input Primitives, please visit the following page: input primitives

This section explores the fundamental concepts that shape user-friendly interactions for input modality.

Interactables should be positioned at a comfortable height to prevent users from constantly looking down, thereby reducing neck strain and enhancing the overall experience. Ensuring a natural line of sight aligns with ergonomic principles promotes longer, more enjoyable sessions without discomfort. For more information, please refer to the comfort page.