If you are new to OpenXR, you may want to read through the OpenXR API Reference by the Khronos Group before continuing with this documentation.
Overview
A Scene Model is composed of anchors. Each anchor represents a unique identifier for an entity, and all data exchange happens with an anchor.
We use OpenXR XrSpace handles to expose access to anchors from within applications. In other words, any operation which involves anchors uses XrSpace handles to identify the affected anchors.
Entity-Component System
We drew inspiration from the Entity-Component-System pattern commonly used within game engines to enable generic anchors to represent various data types. The core idea behind this architecture is to provide specialization via composition instead of inheritance. This strategy allows you to avoid a type explosion in cases where multiple entity types share some characteristics but differ in others.
The system represents each anchor as a spatial entity with a specific set of components that it supports. Each spatial entity is associated with a Universally Unique Identifier (UUID), which you can retrieve using the XrSpace handle of the anchor. Unlike the XrSpace handle that is only active during a session, the anchor’s UUID remains constant through the lifecycle of the anchor. We recommend you use these UUIDs for across-session operations on anchors, such as queries.
Asynchronous Operations and Event Handling
Potentially long-running operations are asynchronous, such as enabling a component and querying anchors. As demonstrated in the code sample below, these methods have corresponding events that signal a status change in the operation. You can use the out-parameter of the request ID to identify the request that triggered the event.
You can handle events using the event loop in the android_main function:
while (androidApp->destroyRequested == 0) {
// Read all pending events.
for (;;) {
int events;
struct android_poll_source* source;
// If the timeout is zero, returns immediately without blocking.
// If the timeout is negative, waits indefinitely until an event appears.
const int timeoutMilliseconds = (app.Resumed == false && app.SessionActive == false && androidApp->destroyRequested == 0) ? -1 : 0;
if (ALooper_pollAll(timeoutMilliseconds, NULL, &events, (void**)&source) < 0) {
break;
}
// Process this event.
if (source != NULL) {
source->process(androidApp, source);
}
}
app.HandleXrEvents();
...
}
The HandleXrEvents function can get quite long. See the ovrApp::HandleXrEvents function in the XrSceneModel sample app for a detailed example.
Locating an Anchor
You locate an anchor using the xrLocateSpace function. This function obtains the pose of a Space relative to a “base space.” In the example below, we get the base space as the view origin (aligned with the headset).
In this example, app.HeadSpace is an XrSpace handle used to store the reference space.
You can obtain local and Stage reference spaces using the XR_REFERENCE_SPACE_TYPE_LOCAL and XR_REFERENCE_SPACE_TYPE_STAGE reference space types. See XrSpaceReferenceType for more information.
Spatial Entity
Extension: XR_FB_spatial_entity
Header include: fb_spatial_entity.h
An anchor will have a specific set of components that it supports. Enabling these components allows access to the operations associated with these components. Component support and state are unique to each anchor.
In general, anchors may support any subset of the following components, provided by the XrSpaceComponentTypeFB enum:
typedef enum XrSpaceComponentTypeFB {
// Works with xrLocateSpace, etc.
XR_SPACE_COMPONENT_TYPE_LOCATABLE_FB = 0,
// enables save, load, erase, etc.
XR_SPACE_COMPONENT_TYPE_STORABLE_FB = 1,
// Bounded 2D component, used in fb_scene extension.
XR_SPACE_COMPONENT_TYPE_BOUNDED_2D_FB = 3,
// Bounded 3D component, used in fb_scene extension.
XR_SPACE_COMPONENT_TYPE_BOUNDED_3D_FB = 4,
// Semantic labels component, used in fb_scene extension.
XR_SPACE_COMPONENT_TYPE_SEMANTIC_LABELS_FB = 5,
// Room layout component, used in fb_scene extension.
XR_SPACE_COMPONENT_TYPE_ROOM_LAYOUT_FB = 6,
// Space container component, used in fb_spatial_entity_container extension.
XR_SPACE_COMPONENT_TYPE_SPACE_CONTAINER_FB = 7,
XR_SPACE_COMPONENT_TYPE_MAX_ENUM_FB = 0x7FFFFFFF
} XrSpaceComponentTypeFB;
Locatable: Enabling this component allows tracking the 6DoF pose of the anchor through the xrLocateSpace method.
Storable: Enabling this component allows the anchor’s persistence (write) operations, which are “save” and “erase.” Scene Model is system-managed, so their anchors cannot be saved or erased from applications. Trying to enable the Storable component for applications will result in an error code.
Bounded_2D: An anchor with this component defines a 2D plane and supports a 2D boundary and a 2D bounding box.
Bounded_3D: An anchor with this component defines a 3D volume and supports a 3D bounding box.
Semantic_labels: An anchor with this component includes semantic labels.
Room_layout: An anchor with this component includes room layout info.
Space_container: An anchor with this component contains a list of spaces.
The xrEnumerateSpaceSupportedComponentsFB function enumerates the supported components for the specified anchor. The return value indicates whether the operation succeeded.
Parameters
space: This parameter contains the handle to the anchor for which you want to enumerate supported components.
componentTypesCapacityInput: The maximum number of component types expected by the caller. The caller uses this value to ensure the array returned through the componentTypes out-parameter does not exceed the bounds of the array allocated by the caller.
[out] componentTypesCountOutput: The number of component types actually enumerated.
[out] componentTypes: An array enumerating the component types supported by the specified anchor.
Returns
A value indicating whether the operation succeeded.
This asynchronous function enables or disables the specified component for the specified anchor. The return value indicates whether the operation succeeded.
Parameters
space: The handle of the anchor for which you want to set the component status.
info: The struct of the component status you wish to enable, as defined below.
[out] requestId: The ID of this asynchronous request.
Returns
A value indicating whether the operation succeeded.
The component status is in the following struct:
componentType: The component for which you wish to set the status.
enable: The value to which you want to set the status.
timeout: A timeout for the operation in nanoseconds. A value of 0.0 indicates no timeout.
NOTE: The API supports enabling Locateable on Scene Model Anchors, except anchors with the Room_layout component. Enabling other components, such as Storable and Bounded_2D, is not supported on Scene Model anchors for applications since the system generates and owns those anchors.
This function gets the current status of the specified component for the specified anchor. You will find the status information in the [out] status parameter.
Parameters
space: The handle to the anchor for the component whose status you wish to query.
componentType: The component type of the component you want to query for status.
[out] status: A struct containing information about the status of the component you queried.
Returns
A value indicating whether the operation succeeded.
You use the xrGetSpaceSemanticLabelsFB function to get one or more semantic labels associated with an anchor, which has the semantic label component enabled.
Parameters
session: The current XrSession.
space: The XrSpace handle to a Scene Model anchor.
[out] semanticLabelsOutput: The buffer where the system will store the semantic labels.
Returns
A value indicating whether the operation was successful.
Semantic Label Struct
typedef struct XrSemanticLabelsFB {
XrStructureType type;
const void* XR_MAY_ALIAS next;
// Input of the label buffer capacity in byte.
uint32_t byteCapacityInput;
// Output of the label buffer size in byte.
uint32_t byteCountOutput;
// Multiple labels represented by raw string, separated by comma (,).
char* labels;
} XrSemanticLabelsFB
The xrGetSpaceSemanticLabelsFB function employs a two-call idiom to get semantic labels in a variable length. The first call passes 0 as byteCapacityInput to retrieve the required buffer size, returned in byteCountOutput. After allocating a buffer at least as large as byteCountOutput, the second call passes a pointer to the allocated buffer to get semantic labels. The system stores the labels of an anchor as a character array in CSV format.
Supported Semantic Labels
Semantic Label
Description
Geometric Representation
Room Structure
CEILING
A ceiling
2D
DOOR_FRAME
A door frame. Must exist within a wall face
2D
FLOOR
A floor
2D
INVISIBLE_WALL_FACE
A wall face added by Space Setup to enclose an open room
2D
WALL_ART
A piece of wall art. Must exist within a wall face
2D
WALL_FACE
A wall face
2D
WINDOW_FRAME
A window frame - must exist within a wall face
2D
Room Contents
COUCH
A couch
2D (the seat) and 3D (the volume)
TABLE
A table
2D (the tabletop) and 3D (the volume)
BED
A bed
2D (the surface) and 3D (the volume)
LAMP
A lamp
3D
PLANT
A plant
3D
SCREEN
A screen
3D
STORAGE
A storage container
2D (a single shelf) and 3D (the volume)
Mesh Objects
GLOBAL_MESH
A triangle mesh of a user’s space captured during Space Setup
Unclassified Objects
OTHER
A general volume
3D
This list of labels is evolving, as we periodically add support for more 2D and 3D objects. Because of this, you should consider the OTHER type as a fallback. It may not be a type in the future, and an object you label as OTHER may need to be changed in the future.
You use the xrGetSpaceBoundary2DFB function to get a 2D polygon boundary associated with an anchor, which has the bounded 2D component enabled.
Parameters
session: The current XrSession.
space: The XrSpace handle to a Scene Model anchor.
[out] boundary2DOutput: The buffer where the system will store the 2D boundary (XrBoundary2DFB).
Returns
A value indicating whether the operation was successful.
The xrGetSpaceBoundary2DFB function employs a two-call idiom, similar to the one used for semantic labels, to retrieve a boundary with a variable number of vertices.
// 2D boundary for two-call idiom with xrGetSpaceBoundary2DFB.
struct XrBoundary2DFB {
XrStructureType type;
const void* XR_MAY_ALIAS next;
// Input, capacity of the vertex buffer.
uint32_t vertexCapacityInput;
// Output, size of the vertex buffer.
uint32_t vertexCountOutput;
// Vertices of the polygonal boundary in the coordinate frame of the associated space.
// Currently only support outer bounds.
XrVector2f* vertices;
} XrBoundary2DFB;
You use the xrGetSpaceRoomLayoutFB function to get the room layout component of an anchor if it is enabled.
Parameters
session: The current XrSession.
space: The XrSpace handle to a Scene Model anchor.
[out] XrRoomLayoutFB: The buffer where the system will store the room layout.
Returns
A value indicating whether the operation was successful.
Room Layout Struct
typedef struct XrRoomLayoutFB {
XrStructureType type;
const void* XR_MAY_ALIAS next;
// Floor of the room layout.
XrUuidEXT floorUuid;
// Ceiling of the room layout.
XrUuidEXT ceilingUuid;
// Input, capacity of wall list buffer.
uint32_t wallUuidCapacityInput;
// Output, number of walls included in the list.
uint32_t wallUuidCountOutput;
// Ordered list of walls of the room layout.
XrUuidEXT* wallUuids;
} XrRoomLayoutFB;
The xrGetSpaceRoomLayoutFB function uses a two-call idiom to retrieve a variable number of walls contained in the room layout component. If no walls exist in the layout, the first call returns a uuidWallsCountOutput parameter of 0. The first call to the function will return the floor and ceiling UUIDs if available.
You use the xrGetSpaceContainerFB function to get a space container associated with an anchor if one is enabled. A space container includes a set of Scene Model anchors identified by their UUIDs.
Parameters
session: The current XrSession.
space: The XrSpace handle to a Scene Model anchor.
[out] spaceContainerOutput: The buffer where the system will store the space container.
Returns
A value indicating whether the operation was successful.
Space Container Struct
typedef struct XrSpaceContainerFB {
XrStructureType type;
const void* XR_MAY_ALIAS next;
// Input, capacity of UUID list.
uint32_t uuidCapacityInput;
// Output, number of spatial entities included in the list.
uint32_t uuidCountOutput;
// List of spatial entities contained in the entity to which this component is attached.
XrUuidEXT* uuids;
} XrSpaceContainerFB;
The xrGetSpaceContainerFB function uses a two-call idiom to retrieve a variable number of spaces included in the container.
You use the xrRequestSceneCaptureFB function to send a request to the system to start a capture flow of a physical environment, such as a room. Once the capture starts, the system will guide the user to complete the capture via system UX flows.
Parameters
session: The current XrSession.
request: Specifies data for a capture request.
[out] requestId: The ID of the asynchronous request.
Returns
A value indicating whether the operation was successful.
XR_TYPE_EVENT_DATA_SCENE_CAPTURE_COMPLETE_FB: Signals the completion of a scene capture.
NOTE: For each XR_TYPE_EVENT_DATA_SCENE_CAPTURE_COMPLETE_FB event, the event data contains an XrEventDataSceneCaptureCompleteFB struct, which includes the result of the scene capture operation.
You perform Scene Model discovery and loading from storage through the anchor query function, which consists of filters that define a search query and an operation to be performed on the search results.
The primary use of the xrQuerySpacesFB function is to find and retrieve anchors from storage. In practice, a reference to an instance of struct XrSpaceQueryInfoFB will be cast to XrSpaceQueryInfoBaseHeaderFB when passing data into the query function and structuring the query filters.
You may use the filter and excludeFilter fields to chain query filters together.
XR_TYPE_EVENT_DATA_SPACE_QUERY_RESULTS_AVAILABLE_FB: Triggers for each space returned by the query.
XR_TYPE_EVENT_DATA_SPACE_QUERY_COMPLETE_FB: Triggers once the system has finished the query operation.
NOTE: For each XR_TYPE_EVENT_DATA_SPACE_QUERY_RESULTS_AVAILABLE_FB event triggered, the two-call idiom xrRetrieveSpaceQueryResultsFB should be used to get XrSpaceQueryResultsFB, which includes a list of query results.
Step 2: An anchor with a room layout component also has a spatial entity container component. In addition to walls, ceiling, and floor, the spatial entity container includes other objects such as tables and couches captured for a room.
To get anchors related to a room layout in the returned anchor:
XrRoomLayoutFB roomLayout = {};
// First call
OXR(FunPtrs.xrGetSpaceRoomLayoutFB(Session, space, &roomLayout));
// Second call
std::vector<XrUuidEXT> wallUuids(roomLayout.wallUuidCountOutput
);
roomLayout.wallUuidCapacityInput = wallUuids.size();
roomLayout.wallUuids = wallUuids.data();
OXR(FunPtrs.xrGetSpaceRoomLayoutFB(Session, space, &roomLayout));
if (isValid(roomLayout.floorUuid)) {
uuidSet.insert(uuidToHexString(roomLayout.floorUuid));
}
if (isValid(roomLayout.ceilingUuid)) {
uuidSet.insert(uuidToHexString(roomLayout.ceilingUuid));
}
for (uint32_t i = 0; i < roomLayout.wallUuidCountOutput; i++) {
uuidSet.insert(uuidToHexString(roomLayout.wallUuids[i]));
}
To get a list of all anchors included in the spatial entity container of the returned anchor:
XrSpaceContainerFB spaceContainer = {};
// First call
OXR(FunPtrs.xrGetSpaceContainerFB(Session, space, &spaceContainer));
// Second call
std::vector<XrUuidEXT> uuids(spaceContainer.uuidCountOutput);
spaceContainer.uuidCapacityInput = uuids.size();
spaceContainer.uuids = uuids.data();
OXR(FunPtrs.xrGetSpaceContainerFB(Session, space, &spaceContainer));
for (uint32_t i = 0; i < spaceContainer.uuidCountOutput; i++) {
uuidSet.insert(uuidToHexString(spaceContainer.uuids[i]));
}
Step 3: Query anchors with a list of collected UUIDs.