Develop
Develop
Select your platform

Inputs and controllers

Updated: May 20, 2025

Overview

Spatial SDK offers seamless integration for using hands and controllers with your Android panel interactions. It forwards the input from your hands or controllers to your Android panels, making it easier to work with inputs.

Interact with input

Use the Controller component to interact with input. This component enables you to read data about a single controller instance.
For example:
class Controller(
    var buttonState: Int = 0,
    var changedButtons: Int = 0,
    var isActive: Boolean = false,
    var type: ControllerType = ControllerType.CONTROLLER,
) : Component
  • type: Set this variable to either ControllerType.CONTROLLER or ControllerType.HAND to specify the type of input you want to use.
  • isActive: This variable is true if a player actively uses the specified input type. For example, if a player uses their hands and the type variable is set to Controller.HAND_TYPE, isActive would be true. If players use their controllers, isActive would be false.
  • buttonState and changedButtons: These variables represent the button states between this tick and the previous one.

Microgestures

Microgestures are a set of hand tracked gestures triggered by your thumb. They are designed to be used in conjunction with other hand tracked gestures, such as pinch and grab, to provide additional input.
Use the MicrogesturesSystem to listen for microgesture events.
// Register a listener and get a handle
val handle = microgesturesSystem.addListener { microgestureBit, isActive ->
    when (microgestureBit) {
        MicrogestureBits.LeftMicrogestureSwipeLeft -> {
            // Handle left hand swipe left
        }
        MicrogestureBits.RightMicrogestureTapThumb -> {
            // Handle right hand thumb tap
        }
        // Handle other microgestures...
    }
}

// Remove the listener when no longer needed
microgesturesSystem.removeListener(handle)

Manage button state

Spatial SDK employs an integer, acting as a bit field, to represent buttonState and changedButtons. Use a bitwise and with the ButtonBits enum to extract information from these values.
For hand inputs, ButtonX and ButtonA represent the left and right index finger pinches, respectively.
// true if the controller is currently pressing the A button
(controller.buttonState and ButtonBits.ButtonA) != 0
// true if the controller is currently holding down either of the triggers
(controller.buttonState and (ButtonBits.ButtonTriggerL or ButtonBits.ButtonTriggerR)) != 0
// a button state representing the buttons that were just pressed down
controller.buttonState and controller.changedButtons
// true if the controller just let go of the X button
(controller.buttonState.inv() and controller.changedButtons and ButtonBits.ButtonX) != 0

Query for controllers

Use the following code to get all entities for the current player with the Controller component.
Query.where{ has(Controller.id) }.eval().filter { it.isLocal() }
This method has a limitation. It returns a Controller component for both hands and controllers, even if only one is active. Use the AvatarBody class to work around this limitation. A local entity with an AvatarBody component gives you access to the head and active hands controllers.
class AvatarBody(
    var head: Entity,
    var leftHand: Entity,
    var rightHand: Entity,
    var isPlayerControlled: Boolean = false,
) : Component

...

// get the current player's `AvatarBody`
val playerBody : AvatarBody = Query.where{ has(AvatarBody.id) }.eval().filter {
    it.isLocal() &&  it.getComponent<AvatarBody>().isPlayerControlled
}.first().getComponent<AvatarBody>()
// get the left controller that is active
val leftController: Controller = playerBody.leftHand.getComponent<Controller>()
// get the position of the user's head
val headPose: Pose = playerBody.head.getComponent<Transform>().transform

Get the position of the controllers

Entities with a Controller component will also have a Transform component. You can get the pose with the following code:
val pose: Pose = playerBody.leftHand.getComponent<Transform>().transform

Respond to button presses on meshes

If you want to execute some code in cases where a controller interacts with a mesh, use the SceneObject.addInputListener() method. For instance, the following code uses SceneObject.addInputListener() to change a sphere’s color when the player points at the sphere and presses the A button:
val red = Color4(1.0f, 0.0f, 0.0f, 1.0f)
val green = Color4(0.0f, 1.0f, 0.0f, 1.0f)
val sphere: Entity = Entity.create(listOf(
    Mesh(Uri.parse("mesh://sphere")),
    Sphere(1.0f),
    Material().apply{ baseColor = red }
))

// later, inside a system
val systemObject =
  systemManager.findSystem<SceneObjectSystem>()?.getSceneObject(entity) ?: return

systemObject.thenAccept{ sceneObject ->
   sceneObject.addInputListener(
    object : InputListener {
        override fun onInput(
            receiver: SceneObject,
            hitInfo: HitInfo,
            sourceOfInput: Entity,
            changed: Int,
            clicked: Int,
            downTime: Long
        ) {
            // check if the A button has changed this tick
            if((changed and ButtonBits.ButtonA) != 0){
                val material = sourceOfInput.getComponent<Material>()
                if((clicked and ButtonBits.ButtonA) != 0) {
                    // set green when first pressing down on the A button
                    material.baseColor = green
                } else {
                    // set red when letting go
                    material.baseColor = red
                }
                sourceOfInput.setComponent(material)
            }
        }
    }
}
As an alternative solution, you can attach an event listener directly to the entity without waiting for the mesh to load.
val red = Color4(1.0f, 0.0f, 0.0f, 1.0f)
val green = Color4(0.0f, 1.0f, 0.0f, 1.0f)
val sphere: Entity = Entity.create(listOf(
    Mesh(Uri.parse("mesh://sphere")),
    Sphere(1.0f),
    Material().apply{ baseColor = red }
))
// can attach directly after creating
sphere.registerEventListener<ButtonReleaseEventArgs>(
        ButtonReleaseEventArgs.EVENT_NAME) { entity, eventArgs ->
          if (eventArgs.button == ControllerButton.A) {
            // make the material green when releasing the A button
            val material = entity.getComponent<Material>()
            material.baseColor = green
            entity.setComponent(material)
          }
        }

Add haptic feedback

Haptics provide tactile feedback to users through the controllers, enhancing immersion and giving helpful feedback. You can send haptic feedback to a specified controller by using the applyHapticFeedback function of the SpatialInterface class. It can be used to simulate various sensations, by adjusting the amplitude, duration, and frequency of feedback. The applyHapticFeedback takes hand, amplitude, duration, and frequency parameters.
  • hand: Specifies which controller the haptic feedback applies to.
  • amplitude: A value between zero and one that represents the strength of the feedback.
  • duration: How long the feedback is applied, in nanoseconds
  • frequency: The frequency of vibration in hertz (Hz).
spatial.applyHapticFeedback(
    if (isLeftHand) Hand.LEFT else Hand.RIGHT, 0.3f, (0.03 * 1e9).toLong(), 128f)
}
In this example, the feedback has an amplitude of 0.3, a duration of 30 milliseconds (converted to nanoseconds), and a frequency of 128 Hz.

Design guidelines

Did you find this page helpful?
Thumbs up icon
Thumbs down icon
Nav Logo
Virtual reality / Meta Horizon
Virtual reality / Meta Horizon
Virtual reality / Meta Horizon
English (US)
© 2025 Meta