Meta Developers

Users may control whether or not the tracker is enabled and what objects they wish to allow applications to have access to. No tracking data is made available to applications without being enabled in the system settings.

At this time, the Meta Quest Dynamic Object Tracker supports the following tracked objects:

Supported Object Classes

Keyboards - includes laptop keyboards, bluetooth keyboards, and wired keyboards. May not detect uncommon keyboard types such as splitboards, unusual color schemes, or keyboards being used on a surface with low visual contrast.

This list may change in future updates as your device becomes even more useful.

Prerequisites

Before you begin working with the Dynamic Object Tracker you will need the following:

Dynamic Object Tracker Prerequisites

A Meta Quest 3 or Meta Quest 3S running the latest version of the Meta Quest Operating System, v72 or newer
Object tracking enabled in the System Settings by the user
The application should enable the XR_META_dynamic_object_tracker extension and any object extensions desired, such as XR_META_dynamic_object_keyboard
The Android manifest file for the application should mention both the USE_ANCHOR_API and USE_SCENE permissions, to use the Dynamic Obect Tracker and access data through Scene apis.
```
<uses-permission android:name="com.oculus.permission.USE_ANCHOR_API" />
<uses-permission android:name="com.oculus.permission.USE_SCENE" />
```
For Developers Only, the logged in Meta account must be registered with a developer organization. This will allow untrusted applications to use spatial data, which is required for this API. Applications downloaded from the Store or App Lab do not have this restriction.

Make sure to use the latest version of the Meta OpenXR Mobile SDK.

Enable Tracking

To enable tracking for supported devices, do the following:

In the headset, go to Settings > Devices > Keyboard
Toggle the slider on this menu to enable Keyboard Tracking. Note that only keyboards are supported at this time, but this menu may change if more objects are available in future releases.

If a physical keyboard is visible in front of you, an outline should now appear in your home environment. With hand tracking enabled, move your hands toward the keyboard to show a passthrough cutout.

Using the Dynamic Object Sample App

With the Dynamic Object Tracker enabled, launch the XrDynamicObjects sample app which is included in the Meta OpenXR Mobile SDK.

To use the tracker in applications that are not listed on the Store or App Lab, you must have a developer organization linked to your Meta account.

Extension: XR_META_dynamic_object_tracker

The XR_META_dynamic_object_tracker OpenXR extension makes available the API that applications may use to interact with the Dynamic Object Tracker. Please read the Prerequisites and Enable Tracking sections before attempting to use this API on your own device.

Checking for Support

To see if the device supports Dynamic Object Tracking, the application should call xrGetSystemProperties with an XrSystemProperties struct correctly typed, and extended with a XrSystemDynamicObjectTrackerPropertiesMETA also correctly typed.

The following functions are available with this extension enabled on supported devices:

Tracker Creation

XrResult xrCreateDynamicObjectTrackerMETA(
    XrSession session,
    const XrDynamicObjectTrackerCreateInfoMETA* createInfo,
    XrDynamicObjectTrackerMETA* dynamicObjectTracker);

This function will immediately return a handle via the XrDynamicObjectTrackerMETA* output parameter, and will asynchronously allocate resources needed for the tracker. The XrDynamicObjectTrackerCreateInfoMETA* is provided for future extension, but currently does not offer any additional configuration. This function gives the system a hint that your application will use the tracker and reserves any needed resources, but does not actually begin tracking until the tracker has been configured.

typedef struct XrEventDataDynamicObjectTrackerCreateResultMETA {
    XrStructureType               type;
    const void* XR_MAY_ALIAS      next;
    XrDynamicObjectTrackerMETA    handle;
    XrResult                      result;
} XrEventDataDynamicObjectTrackerCreateResultMETA;

The result of this function does not indicate a successful tracker bringup; the application should listen for an event of type XR_TYPE_EVENT_DATA_DYNAMIC_OBJECT_TRACKER_CREATE_RESULT_META to confirm whether bringup was successful. If the XrResult in this event is XR_ERROR_INITIALIZATION_FAILED, the tracker has been disabled by the user in the system settings. This state may change throughout the lifetime of an application, so it may make sense for some apps to reattempt tracker creation periodically, like when switching menu screens or when a user toggles the app settings. Similarly, the user may disable the tracker mid-session. When this occurs, no new data will flow through the API, and existing object data will be cleared from memory; the application will not be made aware of this state unless the tracker is destroyed and xrCreateDynamicObjectTrackerMETA is called again.

It is important to verify the handle in the event matches the handle created when the function was called, as you may recieve different handle values for each call to this function. Note that only one instance of the Dynamic Object Tracker is in use at any time, regardless of how many objects are being tracked.

Configuring Tracked Objects

XrResult xrSetDynamicObjectTrackedClassesMETA(
    XrDynamicObjectTrackerMETA dynamicObjectTracker,
    const XrDynamicObjectTrackedClassesSetInfoMETA* setInfo);

You can tell the Dynamic Object Tracker what objects you want to track using the XrDynamicObjectTrackedClassesSetInfoMETA* struct. The xrSetDynamicObjectTrackedClassesMETA function will return XR_ERROR_FEATURE_UNSUPPORTED if any value is passed in that does not map to a supported object type for the current headset. You can check whether an object class is supported by reviewing the enabling extension’s system properties.

typedef struct XrEventDataDynamicObjectSetTrackedClassesResultMETA {
    XrStructureType               type;
    const void* XR_MAY_ALIAS      next;
    XrDynamicObjectTrackerMETA    handle;
    XrResult                      result;
} XrEventDataDynamicObjectSetTrackedClassesResultMETA;

Similar to the xrCreateDynamicObjectTrackerMETA function, this function returns immediately, and a followup event will be sent with the result of the tracker configuration with the type XR_TYPE_EVENT_DATA_DYNAMIC_OBJECT_SET_TRACKED_CLASSES_RESULT_META.

Querying for Tracked Objects

You may query for tracked objects, review their components such as 2d or 3d bounding boxes, and obtain their 3d poses via the Scene API. If you are unfamiliar with Spatial Entities or Scene, you are strongly encouraged to review the Scene Documentation before continuing. You may also reference the XrDynamicObject sample application included in the Mobile SDK for a hands-on learning approach. Dynamically tracked objects can be queried using the xrQuerySpacesFB function, but they do not support the related discover APIs at this time.

XrResult xrGetSpaceDynamicObjectDataMETA(
    XrSpace space,
    XrDynamicObjectDataMETA* dynamicObjectDataOutput);

The Scene API is extended by XR_META_dynamic_object_tracker with a function to retrieve dynamically tracked object data, such as the object type.

Destroying the Tracker

XrResult xrDestroyDynamicObjectTrackerMETA(
    XrDynamicObjectTrackerMETA dynamicObjectTracker);

Applications that are no longer using any dynamically tracked objects are encouraged to call xrDestroyDynamicObjectTrackerMETA on their active tracker handle. This will free up any resources used by the tracker.

Extension: XR_META_dynamic_object_keyboard

The XR_META_dynamic_object_keyboard OpenXR extension enables the device to track Keyboard types, including laptop keyboards, wireless keyboards, and wired keyboards. Only one keyboard is supported by the device at a time. While this tracker is generic, it may have trouble detecting unusual keyboards such as split keyboards, keyboards with unusual color schemes, or keyboards that are on a surface with low contrast, like a white keyboard on a white desk.

Checking for Support

To see if the device supports Keyboard Tracking, the application should call xrGetSystemProperties with an XrSystemProperties struct correctly typed, and extended with a XrSystemDynamicObjectKeyboardPropertiesMETA also correctly typed.

New Enum

The enum XrDynamicObjectClassMETA from XR_META_dynamic_object_tracker is extended with value XR_DYNAMIC_OBJECT_CLASS_KEYBOARD_META.

Did you find this page helpful?