You can find the Unreal Movement sample in the Oculus-VR Unreal Engine Fork.

After completing this section, the developer should:

The Movement sample for Unreal consists of four scenes, each of which demonstrates a slightly different use case. The first two are implemented in the older Actor Component method while the second two utilize the Animation Node method (preferred). The scenes are located in the “Content/Maps” folder.

Face (“Aura”): (“MAP_Aura”) This is a face tracking scene showing how to utilize face tracking and eye tracking using the Actor Component nethod.

Upper Body: (“MAP_HighFidelity”) This is a realistic character with implemented body tracking, face tracking, and eye tracking. This scene utilizes the Oculus rig directly and is not compatible with the Unreal Mannequin.

Retargeting - Mannequin: (“MAP_RetargetMannequinAnimBlueprint”) This scene shows retargetiing options using Mannequin and the Animation Node method. However, the techniques used in this sample can be used for other rigs. This is the most complete example for body tracking.

Face, Eye, and Body Retargeting: (“MAP_HighFidelity_AnimBlueprint”) This sample showcases applying body, face and eye tracking on a realistic character model using the Animation Node method.

The following is the scene select menu shown in the samples to allow you to change between the above scenes.

All samples use a common set of components as described in this section.

In order to switch between levels, you can use the Scene Swtich UI shown in the figure above.

This sample showcases the facial and eye tracking capabilities of the Movement SDK. Aura is our character that is modeled to have slightly stylistic and fantastical features; larger than normal eyes and floral petals for hair. Because Aura’s proportions are slightly different from a human, the facial tracking data needs to be adjusted to fit the character and constrained to ensure the character doesn’t clip through her own geometry when moving or animating. This is achieved by setting constraints on how the facial data is applied to the character, using facial expression modifiers. Aura has free floating head and hands which means applying a 3-point body tracking (head, 2x hands) will produce convincing results since there are no elbows or knees to worry about.

Aura is retargeted using the Actor Component implementation. By opening the map and inspecting the pawn you can see that she has the following components added:

Additionally she has the following components:

These are components that all improve the tracking in various ways. They live within the sample and you can either use them for inspiration for achieving similar functionality or you can migrate them over to your project.

The different characters’ faces might not always align with the physical face correctly which is why face Expression Modifiers are needed. The face morph targets can be modified by either completely disabling them or increasing/decreasing the effect of what the physical face does. For example, when the Aura character smiles, her lips can appear behind her teeth, but you can rectify this by using a modifier for the lips.

The face Expression Modifiers are enabled in the Face Tracking component. They are an array of 33 elements.

An individual element is shown below.

This sample showcases how to use the Movement SDK with a half body character. This is our 3D scanned character and has realistic proportions. This means the tracking data can be applied directly onto the character. Note that we need to apply Twist Distribution to the arms to make sure the arms don’t twist in an unnatural way.

The model is set up using the Actor Component implementation using the following features:

Additionally he has the following components:

This sample showcases applying body tracking to the Unreal 5 Mannequin rig, using the OculusXRMovement Body Tracking animation node. Since the Unreal Mannequin skeleton is different from the Oculus Body Tracking Skeleton, a bonemap is needed to map the tracking skeleton names to the target skeleton bones. The OculusXRMovement Body Tracking node supports Upper and Full Body tracking. Through the UI panel on the side, you can select between the two modes in runtime. When the sample is using upper body tracking we have no information about the legs. Instead Foot Grounding keeps the feet on the floor. It is the process of running traces down to the ground and using IK to place the feet there so that it appears natural.

Mannequin is set up using the Animation Blueprint implementation using the following features:

Additionally, this character has the following components:

This sample showcases applying body, face and eye tracking on this sample, hence full tracking. Compared to the sample mentioned earlier, this example uses the Animation Node implementation of body, face and eye Tracking.

This model is set up using the Animation Blueprint implementation using the following features:

Additionally this sample has the following components:

This section outlines all blueprints in the sample project and explains their purpose:

To achieve correct bone/joint rotation and not make body parts look like candy-wrappers, add a Twist Distribution component to adjust the joints. (This is not needed when using Animation Node implementation.) This can be seen in hand rotating where the elbow bone and the wrist bone are rotated but the joints’ rotation is not correctly distributed which leads to candy-wrapper look:

Without twist distribution:

With twist distribution:

The simulated body’s bone spacing of the spine and arms can change after updating the body state. The original spacings are recorded at the Start and are used to correct the spacing after updating the body state. The Deformation Logic component adjusts these spacings and it has the option to skip the Head or Hips for specific reasons.

You can use AC_AndroidPermissions in your project to help you request permissions on the device:

You can take a look at our BP_MovementSampleGameMode to see how this can be setup.