In this page you will learn more about how the Scene Model is implemented, how it is captured through Space Setup, and how Scene Anchors provide access to the real world environment data.

The Scene Model is a single comprehensive representation of the physical environment. It provides geometric and semantic information that can easily be indexed and queried by an application.

Space Setup is a system managed process that guides a user through the capture flow, capturing a single Scene Model that is accessible to all apps which have been granted access by the user.

See Scene Overview for more foundational information about the Scene Model and Space Setup.

The Scene Model is composed of a number of Scene Anchors, each of which holds some further data describing its intent. The anchor framework is closely related to the Entity-Component-System pattern, whereby an Entity is little more than a storable data type with a unique identifier, Components contain data and are stored on Entities, and Systems operate globally on all Entities that have the necessary Components. In the context of the Scene Model, a Scene Anchor is an Entity, and can have any number of Components on it (such as semantic classification, 2D bounding plane, locatable).

Space Setup (formerly known as Scene Capture) is the process which captures a Scene Model. It is managed by the system so that all apps running on a device have access to the same environment data. This is in contrast to the paradigm where every app would need to scan the environment during the its lifecycle. Space Setup is a user-guided process: it first scans the environment to obtain a space mesh and to extract the space information (such as floor and ceiling height, walls, objects), and then the user completes the process by correcting any errors (wall positioning) and adding missing information (room objects). The process can be invoked by the system or by an app.

The Scene Model contains sensitive user data and is thus fully controlled by the user for whether an app can access the data or not. Apps must declare their intention of using the permission through the app’s manifest file, and then perform a permission request at runtime to prompt the user for approval.

Scene Anchors require components in order for them to describe the environment represented by the Scene Model. In order for a component to provide data, it has to be enabled. Apps must therefore query both if an Scene Anchor supports a given component, and if the component has been enabled.

As Scene Anchors can only be created through the Space Setup process, we call these types of anchors system-managed, while Spatial Anchors are user-managed.

Scene anchors are created by Meta Horizon OS during Space Setup, while spatial anchors are created by your application. Scene anchors contain other information specific to the scene, such as the anchor’s pose. Finally, your app can only create spatial anchors, but it can query scene anchors.

A Locatable component informs the system that this anchor can be tracked. Once enabled, an app can continually query the pose information of the anchor, which can vary due to a difference in tracking accuracy over an anchor’s life.

A RoomLayout component contains references to anchors that make up the walls, the ceiling and the floor. An AnchorContainer component contains a reference to a list of child anchors.

The Bounded2D and Bounded3D components provide information about the dimensions of an anchor. They have a size property which captures the dimensions, and an offset property which describes the difference between the origin of the 2D/3D bounding box and the anchor’s origin defined by the Locatable component. The TriangleMesh component provides an indexed triangle mesh for an anchor. The Boundary2D component provides access to the polygon outline of an anchor, commonly found on floor anchors.

The SemanticClassification component categorizes the anchor into a number of classifications. See below for the complete list of possible classifications.

Both Storable and Shareable components only apply to Spatial Anchors.

As the system manages Scene Anchors, the supported components are predetermined.

The Scene Anchor for the room will have: a RoomLayout component to refer to the ceiling, walls and floor; an AnchorContainer component which holds all the Scene Anchors of the room.

Scene Anchors for 2D elements (such as walls, ceilings, floors, windows and wall art) have: a Locatable component to get the position of the anchor; a Semantic Classification component for the labels; a Bounded2D component for the bounding box dimensions; and optionally a Boundary2D component for a polygon outline (floors use this component to show the outline of a room made from the walls).

Scene Anchors that are 3D-only (such as shelves, screens, plants, and other) are similar to 2D Scene Anchors, but have a Bounded3D component instead of a Bounded2D.

Some Scene Anchors are both 2D and 3D (such as tables and couches), where the 3D component refers to the bounding volume, and the 2D component corresponds to an area of interest, known as the functional surface. These anchors are similar to 2D anchors, but also contain a Bounded3D component.

The Scene Mesh is a triangle mesh that covers the entire space with a single static artifact. It is snapped to the surface near the room layout elements (such as walls, ceiling and floor), and it approximately describes the boundaries of all other objects in the space. It is represented as a common Scene Anchor. However, there is only a single instance of this object per space.

The Scene Anchor for a Scene Mesh will have a Locatable component to get the position of the anchor, a Semantic Classification component for the label GLOBAL_MESH, and a TriangleMesh component to provide the geometry.

Scene Anchors are defined in a Cartesian right-handed coordinate system to match the default OpenXR coordinate system, while Unity uses a left-handed coordinate system. This results in both design-time considerations for spawning objects that need to match the orientation and dimensions of Scene Anchors, and a runtime conversion of each 3D position (mostly hidden to the developer).

Semantic classifications categorize Scene Anchors into a predetermined and system-managed list of object types. Semantic classes separate objects beyond their basic geometric description to provide an app developer with the possibility of applying classification-specific game logic.

This list of labels is evolving, as we periodically add support for more 2D and 3D objects. Because of this, you should consider the OTHER type as a fallback. It may not be a type in the future, and an object you label as OTHER may need to be changed in the future.

Now that you’ve learned how the Scene Model and Space Setup/Scene Capture works, it’s time to access the data in Unity.