Spatial SDK Architecture

Spatial SDK is a framework that provides XR capabilities such as graphics, input, and audio. Spatial SDK integrates with common and critical Android frameworks like Android UI and JetPack Compose. The framework makes it easy for developers familiar with Android to build immersive experiences without learning a new set of tools or frameworks. By leveraging existing Android knowledge, Spatial SDK can provide a more intuitive development experience while delivering powerful XR capabilities.

Spatial SDK’s architecture fits within the existing Android ecosystem, meaning that SDKs components sit on top of Android components like Activities, Fragments, and Views. The SDK leverages Android’s strengths and augments them to offer a unique set of XR capabilities.

Spatial SDK comprises several core modules that work in tandem to offer XR capabilities. These modules include graphics, input, and audio. Each module is independently usable, allowing you to select the features needed for your specific application.

Creating a new activity for XR is recommended. This activity should inherit from AppSystemActivity. You can also reference multiple activities if needed. For instance, you might have a main activity that hosts a 3D scene, and another that displays a settings menu. Creating a new activity for each screen helps organize your code and create a more modular application. Additionally, referencing multiple activities enables code reuse beyond Meta Quest devices.

The core of the Spatial SDK is the Entity-Component-System (ECS) architecture pattern. ECS separates data and execution. Entities can have multiple components as well as systems that operate on those components. This is a software pattern that uses composition over inheritance.

Entities represent the objects in your application. They are lightweight containers that group related components together. For example, a player entity might have transform, health, and score components. Components hold the data for a specific aspect of an entity.

Components can hold simple or complex types of data, and they must be serializable. For example, a position component might hold x, y, z coordinates as floats for an entity.

Systems execute on data in your app. Systems use queries to retrieve the data they need to operate on. For example, a collision detection system might query all entities with a collidable component and check their positions to determine if any collisions have occurred.

This architecture pattern allows for greater composition, flexibility, and modularity in your code, making it easier to maintain and scale.

The DataModel is fundamental to Spatial SDK’s design. It is a key-value store for caching and storing data. Queries retrieve data from the DataModel. The DataModel is flexible and scalable, allowing you to store and retrieve data in various ways. Additionally, the DataModel retrieves and manipulates data within its entities and components. Using this key-value store, you can cache data for faster access and store data in a way that is easy to query.

The DataModel provides first-class replication support. You can use this support to create multiplayer applications, like a meeting application that supports Avatars and VOIP.

Queries allow you to retrieve specific data from the DataModel based on certain criteria. For example, you might query all entities with a player tag to retrieve a list of all players in your application. Queries can be simple or complex, allowing you to retrieve the data needed for your applications. By using queries, you can filter data, making it easier to execute only the relevant information for the system. Queries are powerful tools that help you build more dynamic and responsive applications.

The Environment represents the 3D world where your application takes place, while Objects are the interactive elements within that world.

The 3D representation of an object in your environment is known as a SceneMesh. glTF files are the primary way to set up a SceneMesh. These files contain information about the object’s geometry, materials, and animations. Using glTF files, you can import and export 3D models into your applications.

An instance of a SceneMesh is known as a SceneObject. You create SceneObjects via the Creator Systems from the DataModel. You can create and manage multiple instances of the same SceneMesh within your application to create more complex and dynamic scenes.

Panels represent 2D UI elements positioned in 3D space, allowing you to create interactive and immersive user interfaces within your 3D environment.

PanelSceneObject is the 3D scene object responsible for displaying the 2D UI. This object allows you to create interactive and immersive user interfaces within your 3D environment. By combining 2D UI elements with 3D objects, you can create more engaging and intuitive user experiences.

Spatial SDK is neutral about the UI technology you use. The close collaboration between the Panel system and the DataModel matters more. Spatial SDK supports Android UI and Jetpack Compose. You can also integrate React Native. You can implement an activity based on Spatial SDK in addition to your current mobile applications, allowing you to gradually adopt XR capabilities into your existing app.

Spatial SDK can host Panels within the same activity as your 3D Environment and Objects. Spatial SDK can also host 2D activities, allowing the Spatial SDK activity to work alongside apps to allow for gradual adoption. By using activities, you can create more modular and maintainable code while still leveraging Spatial SDK’s XR capabilities.

You cannot use multiple instances of the AppSystemActivity within a single app.

Interaction SDK converts controller input into interactions on the 2D panel. Spatial SDK supports hands and controllers. By supporting multiple input methods, Spatial SDK makes it easier for you to create accessible and intuitive user interfaces.

The User represents the person experiencing your application through hand tracking, gaze input, and controllers. Spatial SDK provides comprehensive support for user interactions within the 3D environment.

Spatial SDK automatically handles user positioning within the 3D environment, tracking head movement and orientation to maintain proper spatial relationships with Environment, Objects, and Panels.

Spatial SDK supports multiple input modalities to accommodate different user preferences and interaction patterns:

These input methods are managed through the VR feature system, which automatically enables appropriate tracking capabilities based on the hardware and user preferences.