Meta Developers

Controller input and tracking enables the player to interact with the world around them. In this guide, you will create a basic interaction that enables the player to grab objects in the world and manipulate them.

Prerequisites

This guide assumes you have a cusomt Pawn Blueprint class created with appropriate motion controller components set up on it. See Getting Started with Controller Input and Tracking for a guide on setting that up.

Access and Store Controller Tracking Data

Use the Tick event to access and store the controller tracking data each frame. You can use the stored data in other locations without having to execute the logic to access it over and over, which saves time during development and makes it more efficient at runtime.

Drag the OculusXRController component for the right hand from the Components section of the My Blueprint panel to the Event Graph next to the Tick event. Select Get RightOculusXRController.
Drag off the output pin of the RightOculusXRController node and choose Get Motion Controller Data.
On the Get Motion Controller Data function, set the Hand pin value to Right.
Drag a connection from the output exec pin of the Tick event to the input exec pin of the Get Motion Controller Data function.
Drag off the Motion Controller Data output pin of the Get Motion Controller Data function and choose Promote to Variable.
In the Variables section of the My Blueprint panel, rename the new variable to Right Motion Controller Data.
Drag off the output pin of the Set node and choose Break XRMotionControllerData.
Click the down arrow on the Break XRMotionControllerData node to expose all of the output pins.

Trace a Ray from the Controller

You want the user to be able to interact with the environment using the controller. To detect what the controller is aiming at, trace a ray out from the controller.

Drag off the Aim Rotation output pin of the Break XRMotionControllerData node and choose Vector Forward.
Drag off the Return Value output pin of the Vector Forward function and choose Multiply.
Right-click on the empty input pin of the Multiply node and choose To Float (Single-Precision) to use a single value to multiply the input Vector by. Set the value to 5000.0.
Drag off the output exec pin of the Set node and choose Line Trace By Channel.
Drag a connection from the Aim Position output pin of the Break XRMotionControllerData node to the Start input node of the Line Trace By Channel function. Then, drag a connection from the output pin of the Multiply node to the End input node of the Line Trace By Channel function.
Drag off the output exec pin of the Line Trace By Channel function and choose Branch. Then, drag a connection from the Return Value output pin of the Line Trace By Channel function to the Condition input pin of the Branch node.
Drag off the Out Hit output pin of the Line Trace By Channel function and choose Break Hit Result. Now, click the down arrow on the Break Hit Result node to display all of the output pins.
Drag off the Hit Actor output pin and choose Promote to Variable. In the My Blueprint panel, rename the new variable to TargetedActor. This will be used throughout the logic to reference the Actor being targeted or interacted with.
Drag a connection from the True exec pin of the Branch node to the input exec pin of the Set node.
Drag off the output exec pin of the Set node and choose Print String. Then, drag a connection from the output pin of the Set node to the In String input pin of the Print String node.
In the Blueprint Editor toolbar, click the Compile button to compile your Pawn class. Click the Save button to save the asset.
Run the level on your device to view the name of the Actor the controller is pointing at printed to the screen as you move the controller around.

Add Visuals for Ray and Reticle

Delete the Print String and Get Display Name function as those were only needed to confirm the confirm the Actor being targeted.
In the Components panel, add a new Sphere component under the RightOculusXRController component. Name the component Ray.
With the Ray component selected, go to the Details panel and set the Location to (X=25.000000,Y=0.000000,Z=0.000000) and the Scale to (X=0.500000,Y=0.001000,Z=0.001000). The skewed scale causes the sphere to appear more like a small cylinder that tapers off as it goes away fro mthe contorller. The Sphere component’s origin is at the center of the sphere so adjusting the location offsets it to start from the controller location instead of being centered on it.
Set the following properties in the Collision section as well:
- Generate Overlap Events: False
- Can Character Step Up On: No
- Collision Presets: NoCollision
In the Components panel, add a new Sphere component under the RightOculusXRController component. Name the component Reticle.
With the Reticle component selected, go to the Details panel and set the Scale to (X=0.010000,Y=0.010000,Z=0.010000).
Set the following properties in the Collision section as well:
- Generate Overlap Events: False
- Can Character Step Up On: No
- Collision Presets: NoCollision
Drag the Reticle component from the Components tab into the Event Graph next to the Break Hit Result node.
Drag off the output pin of the Reticle node and choose Set World Location.
Drag a connection from the output exec pin of the Set node to the input exec pin of the Set World Location function.
Drag a connection from the Impact Point output pin of the Break Hit Result node to the New Location input pin of the Set World Location function.
In the Blueprint Editor toolbar, click the Compile button to compile your Pawn class. Click the Save button to save the asset.
Run the level on your device to view the reticle on Actors as you move the controller around.

To take this concept further and polish the experience, consider the following:

Apply an unlit, semi-transparent material to the reticle and ray components.
Adjust the scale of the reticle depending on the distance to the targeted Actor, making it larger as it gets farther away. Apply a minumum and maximum scale to ensure it never gets too large or too small.
Adjust the length of the ray visual depending on the distance from the controller to anything in front of it.
Add a small haptic effect when the controller targets a different Actor. You can compare the TargetedActor variable to the Hit Actor output pin to determine when a different Actor is being targeted.
Create a Haptic Effect Curve asset, set up the curved for the effect, and play it in the Event Graph when the TargetedActor changes.

Select Actor to Interact With

Right-click in the Event Graph and search for the name of the Input Action you created for the A button on the right controller. In this example it is XR_R_A_InputAction. Choose the Enhanced Action Event for the Input Action.
In the My Blueprint panel, create a new Actor Reference variable named ControlledActor.
Drag the new variable into the Event Graph and choose Set ControlledActor.
Drag the TargetedActor variable into the Event Graph and choose Get TargetedActor.
Drag a connection from the output pin of the TargetedActor node to the Controlled Actor input pin of the Set node.
Drag off the output exec pin of the Set node and choose Print String.
Drag a connection from the output pin of the Set node to the In String input pin of the Print String function.
In the Blueprint Editor toolbar, click the Compile button to compile your Pawn class. Click the Save button to save the asset.
Run the level on your device to view the name of the Actor saved in the ControlledActor variable printed to the screen as you press the A button while pointing at an Actor.

Manipulate the Selected Actor

Delete the Print String function as that was only needed to confirm the Actor being selected.
Right-click in the Event Graph and search for the name of the Input Action you created for the Grip Axis on the right controller. In this example it is XR_R_GripAxis_InputAction. Choose the Enhanced Action Event for the Input Action.
Drag the Right Motion Controller Data variable from the My Blueprint panel into the Event Graph. Choose Get Right Motion Controller Data.
Drag off the output pin of the Right Motion Controller Data node and choose Break XRMotionControllerData. Click the down arrow on the Break XRMotionControllerData node to show all properties.
Drag off the Triggered exec pin of the EnhancedInputAction event and choose Set Actor Location and Rotation.
Drag the ControlledActor variable from the My Blueprint panel into the Event Graph. Choose Get ControlledActor.
Drag a connection from the output pin of the ControlledActor pin to the Target input pin of the Set Actor Location and Rotation function.
Drag a connection from the Grip Position output pin of the Break XRMotionControllerData node to the New Location input pin of the Set Actor Location and Rotation function.
Drag a connection from the Grip Rotation output pin of the Break XRMotionControllerData node to the New Rotation input pin of the Set Actor Location and Rotation function.
In the Blueprint Editor toolbar, click the Compile button to compile your Pawn class. Click the Save button to save the asset.
Run the level on your device and press the A button to select an Actor. Then, squeeze the Grip to manipulate the selected Actor.

To take this concept further and polish the experience, consider the following:

Interpolate the ControlledActor from its location to the controller when you squeeze the grip.
Enable physics on Actors when you release the grip to drop the item to the floor.

Learn more

To learn more about using controllers in XR applications in Unreal Engine, see the following guides:

Did you find this page helpful?