Transitioning to 3D development

• Pose: A combination of a position (Vector3 for x, y, z coordinates) and a rotation (Quaternion for w, x, y, z). Poses describe where something is and which direction it faces in 3D space.
• Quaternion: A mathematical representation of 3D rotation that avoids gimbal lock (a problem where you lose a degree of rotational freedom). You don’t need to understand the math — use helper methods like Quaternion.lookRotation() to create them.
• Reference space: Defines the origin point for tracking. LOCAL_FLOOR places the origin at floor level beneath the user’s initial position, which is the most common choice.
• Entity: A lightweight identifier that groups related components together in the ECS architecture. See ECS overview.
• Component: A data container attached to an entity (for example, Transform, Mesh, Material). Components define what an entity is, not what it does.
• System: Code that runs each frame and processes entities with specific components. Systems define behavior.
• Panel: A 2D UI surface (built with Android Views or Jetpack Compose) that is positioned in 3D space.
• Passthrough: A mode where the headset’s cameras show the real world with virtual content overlaid on top, creating a mixed reality experience.
• IBL (Image-Based Lighting): A technique that uses an environment map image to light the scene realistically, simulating how light bounces off surfaces in the real world.

• Controller API: Provides tracking data about both physical controllers and hands.
• AvatarBody API and AvatarAttachment: Provides body tracking data.
  This code sample uses the Controller and AvatarBody APIs to detect if the user is pressing the menu button on their left controller.

    val leftController =
      Query.where { has(AvatarBody.id) }.filter { isLocal() and by(AvatarBody.isPlayerControlledData).isEqualTo(true) }
        .eval()
        .first()
        .getComponent<AvatarBody>()
        .leftHand
        .tryGetComponent<Controller>()
  
    if (leftController != null && leftController.getPressed(ButtonBits.ButtonMenu)) {
        onMenuButtonPressed()
    }
  
  
    if (leftController != null && leftController.getReleased(ButtonBits.ButtonMenu)) {
        Log.i("MrukInputSystem", "Menu button released")
    }
  
    /** returns true if the button was pressed this frame */
    fun Controller.getPressed(button: Int): Boolean {
    return (buttonState and changedButtons and button) != 0
    }
  
    /** returns true if the button was released this frame */
    fun Controller.getReleased(button: Int): Boolean {
        return buttonState and button == 0 && changedButtons and button != 0
    }
  
• Scene API: Provides controller and head pose data as well as multiple methods to affect the overall scene.
  This code sample uses the Scene API to adjust the environment and move the user to the world origin.

    class MyActivity : AppSystemActivity() {
        override fun onSceneReady() {
            super.onSceneReady()
  
            // Set the reference space for tracking
            scene.setReferenceSpace(ReferenceSpace.LOCAL_FLOOR)
  
            // Enable/disable passthrough for MR mode
            val isMrMode = scene.isSystemPassthroughEnabled()
  
            // Set view origin position
            scene.setViewOrigin(0f, 0.0f, 2.0f, 0.0f)
  
            // Update environment lighting
            scene.updateIBLEnvironment("chromatic.env")
        }
    }
  

// Function to get the pose of the user's head
fun getHeadPose(): Pose {
  val head =
      Query.where { has(AvatarAttachment.id) }
          .eval()
          .filter { it.isLocal() && it.getComponent<AvatarAttachment>().type == "head" }
          .first()
  return head.getComponent<Transform>().transform
}