Meta Developers

Spatial SDK offers seamless integration for using hands and controllers with your Android panel interactions. It forwards the input from your hands or controllers to your Android panels, making it easier to work with inputs.

The Controller component

Use the Controller component to interact with input. This component enables you to read data about a single controller instance.

For example:

class Controller(
    var buttonState: Int = 0,
    var changedButtons: Int = 0,
    var isActive: Boolean = false,
    var type: ControllerType = ControllerType.CONTROLLER,
) : Component

type: Set this variable to either ControllerType.CONTROLLER or ControllerType.HAND to specify the type of input you want to use.
isActive: This variable is true if a player actively uses the specified input type. For example, if a player uses their hands and the type variable is set to Controller.HAND_TYPE, isActive would be true. If players use their controllers, isActive would be false.
buttonState and changedButtons: These variables represent the button states between this tick and the previous one.

Manage button state

Spatial SDK employs an integer, acting as a bit field, to represent buttonState and changedButtons. Use a bitwise and with the ButtonBits enum to extract information from these values.

For hand inputs, ButtonX and ButtonA represent the left and right index finger pinches, respectively.

// true if the controller is currently pressing the A button
(controller.buttonState and ButtonBits.ButtonA) != 0
// true if the controller is currently holding down either of the triggers
(controller.buttonState and (ButtonBits.ButtonTriggerL or ButtonBits.ButtonTriggerR)) != 0
// a button state representing the buttons that were just pressed down
controller.buttonState and controller.changedButtons
// true if the controller just let go of the X button
(controller.buttonState.inv() and controller.changedButtons and ButtonBits.ButtonX) != 0

Query for controllers

Use the following code to get all entities for the current player with the Controller component.

Query.where{ has(Controller.id) }.eval().filter { it.isLocal() }

This method has a limitation. It returns a Controller component for both hands and controllers, even if only one is active. Use the AvatarBody class to work around this limitation. A local entity with an AvatarBody component gives you access to the head and active hands controllers.

class AvatarBody(
    var head: Entity,
    var leftHand: Entity,
    var rightHand: Entity,
    var isPlayerControlled: Boolean = false,
) : Component

...

// get the current player's `AvatarBody`
val playerBody : AvatarBody = Query.where{ has(AvatarBody.id) }.eval().filter {
    it.isLocal() &&  it.getComponent<AvatarBody>().isPlayerControlled
}.first().getComponent<AvatarBody>()
// get the left controller that is active
val leftController: Controller = playerBody.leftHand.getComponent<Controller>()
// get the position of the user's head
val headPose: Pose = playerBody.head.getComponent<Transform>().transform

Get the position of the controllers

Entities with a Controller component will also have a Transform component. You can get the pose with the following code:

val pose: Pose = playerBody.leftHand.getComponent<Transform>().transform

Respond to button presses on meshes

If you want to execute some code in cases where a controller interacts with a mesh, use the SceneObject.addInputListener() method. For instance, the following code uses SceneObject.addInputListener() to change a sphere’s color when the player points at the sphere and presses the A button:

val red = Color4(1.0f, 0.0f, 0.0f, 1.0f)
val green = Color4(0.0f, 1.0f, 0.0f, 1.0f)
val sphere: Entity = Entity.create(listOf(
    Mesh(Uri.parse("mesh://sphere")),
    Sphere(1.0f),
    Material().apply{ baseColor = red }
))

// later, inside a system
val systemObject =
  systemManager.findSystem<SceneObjectSystem>()?.getSceneObject(entity) ?: return

systemObject.thenAccept{ sceneObject ->
   sceneObject.addInputListener(
    object : InputListener {
        override fun onInput(
            receiver: SceneObject,
            hitInfo: HitInfo,
            sourceOfInput: Entity,
            changed: Int,
            clicked: Int,
            downTime: Long
        ) {
            // check if the A button has changed this tick
            if((changed and ButtonBits.ButtonA) != 0){
                val material = sourceOfInput.getComponent<Material>()
                if((clicked and ButtonBits.ButtonA) != 0) {
                    // set green when first pressing down on the A button
                    material.baseColor = green
                } else {
                    // set red when letting go
                    material.baseColor = red
                }
                sourceOfInput.setComponent(material)
            }
        }
    }
}

As an alternative solution, you can attach an event listener directly to the entity without waiting for the mesh to load.

val red = Color4(1.0f, 0.0f, 0.0f, 1.0f)
val green = Color4(0.0f, 1.0f, 0.0f, 1.0f)
val sphere: Entity = Entity.create(listOf(
    Mesh(Uri.parse("mesh://sphere")),
    Sphere(1.0f),
    Material().apply{ baseColor = red }
))
// can attach directly after creating
sphere.registerEventListener<ButtonReleaseEventArgs>(
        ButtonReleaseEventArgs.EVENT_NAME) { entity, eventArgs ->
          if (eventArgs.button == ControllerButton.ButtonA) {
            // make the material green when releasing the A button
            val material = entity.getComponent<Material>()
            material.baseColor = green
            entity.setComponent(material)
          }
        }

Did you find this page helpful?