Spatial SDK Architecture

Spatial SDK is a framework that provides XR capabilities such as graphics, input, and audio. It integrates with Android frameworks like Android UI and Jetpack Compose. Developers familiar with Android can build immersive experiences without learning new tools. The SDK builds on existing Android knowledge to deliver XR capabilities.

Spatial SDK’s architecture fits within the existing Android ecosystem. The SDK’s components sit on top of Android components like Activities, Fragments, and Views, adding XR capabilities to Android’s strengths.

If you’re coming from Android development, the following table shows how familiar Android concepts relate to their Spatial SDK equivalents:

Spatial SDK uses a right-handed coordinate system. The positive X axis points to the right, the positive Y axis points up, and the negative Z axis points forward (into the screen). All distance values are in meters. For example, Vector3(1f, 0f, 0f) represents a point 1 meter to the right of the origin.

Rotations are represented as quaternions (Quaternion), which encode 3D rotations without gimbal lock. A Pose combines a position (Vector3) and a rotation (Quaternion) to describe both where something is and which direction it faces.

Spatial SDK comprises several core modules that work in tandem to offer XR capabilities. These modules include graphics, input, and audio. Each module is independently usable, allowing you to select the features needed for your specific application.

Creating a new activity for XR is recommended. This activity should inherit from AppSystemActivity. You can also reference multiple activities if needed. For instance, you might have a main activity that hosts a 3D scene, and another that displays a settings menu. Creating a new activity for each screen helps organize your code and create a more modular application. Additionally, referencing multiple activities enables code reuse beyond Meta Quest devices.

The core of the Spatial SDK is the Entity-Component-System (ECS) architecture pattern. ECS separates data and execution. Entities can have multiple components as well as systems that operate on those components. This is a software pattern that uses composition over inheritance.

Entities represent the objects in your application. They are lightweight containers that group related components together. For example, a player entity might have transform, health, and score components. Components hold the data for a specific aspect of an entity.

Components can hold simple or complex types of data, and they must be serializable. For example, a position component might hold x, y, z coordinates as floats for an entity.

Systems execute on data in your app. Systems use queries to retrieve the data they need to operate on. For example, a collision detection system might query all entities with a collidable component and check their positions to determine if any collisions have occurred.

This architecture pattern allows for greater composition, flexibility, and modularity in your code, making it easier to maintain and scale.

The DataModel is fundamental to Spatial SDK’s design. It is a key-value store for caching and storing data. Queries retrieve data from the DataModel. The DataModel retrieves and manipulates data within its entities and components. You can cache data for faster access and store it in a way that is efficient to query.

The DataModel provides first-class replication support. You can use this support to create multiplayer applications, like a meeting application that supports Avatars and VOIP.

Queries retrieve specific data from the DataModel based on criteria you define. For example, you can query all entities with a player tag to get a list of all players. Queries filter data so that systems process only relevant entities.

The Environment represents the 3D world where your application takes place, while Objects are the interactive elements within that world.

The 3D representation of an object in your environment is known as a SceneMesh. glTF files are the primary way to set up a SceneMesh. These files contain information about the object’s geometry, materials, and animations. Using glTF files, you can import and export 3D models into your applications.

An instance of a SceneMesh is known as a SceneObject. You create SceneObjects via the Creator Systems from the DataModel. You can create and manage multiple instances of the same SceneMesh within your application to create more complex and dynamic scenes.

Panels represent 2D UI elements positioned in 3D space, allowing you to create interactive and immersive user interfaces within your 3D environment.

PanelSceneObject is the 3D scene object responsible for displaying the 2D UI. This object allows you to create interactive and immersive user interfaces within your 3D environment. By combining 2D UI elements with 3D objects, you can create more engaging and intuitive user experiences.

Spatial SDK is neutral about the UI technology you use. The close collaboration between the Panel system and the DataModel matters more. Spatial SDK supports Android UI and Jetpack Compose. You can also integrate React Native. You can implement an activity based on Spatial SDK in addition to your current mobile applications, allowing you to gradually adopt XR capabilities into your existing app.

Spatial SDK can host Panels within the same activity as your 3D Environment and Objects. Spatial SDK can also host 2D activities, allowing the Spatial SDK activity to work alongside apps to allow for gradual adoption. By using activities, you can create more modular and maintainable code while still leveraging Spatial SDK’s XR capabilities.

You cannot use multiple instances of the AppSystemActivity within a single app.

Interaction SDK converts controller input into interactions on the 2D panel. Spatial SDK supports hands and controllers. By supporting multiple input methods, Spatial SDK makes it easier for you to create accessible and intuitive user interfaces.

The User represents the person experiencing your application through hand tracking, eye and head tracking, and controllers. Spatial SDK provides comprehensive support for user interactions within the 3D environment.

Spatial SDK automatically handles user positioning within the 3D environment, tracking head movement and orientation to maintain proper spatial relationships with Environment, Objects, and Panels.

Spatial SDK supports multiple input modalities to accommodate different user preferences and interaction patterns:

These input methods are managed through the VR feature system, which automatically enables appropriate tracking capabilities based on the hardware and user preferences.