Meta Developers

Health and Safety Recommendation: While building mixed reality experiences, we highly recommend that you evaluate your content to offer your users a comfortable and safe experience. Please refer to the Health and Safety and Design guidelines before designing and developing your app using Depth.

Overview

What is Depth API?

The Depth API provides real-time depth maps that apps can use to sense the environment. Primarily, it enhances mixed reality (MR) by allowing virtual objects to be occluded by real-world objects and surfaces, which makes them appear integrated into the actual environment. Occlusion is crucial because it prevents virtual content from appearing as a layer over the real world, which can disrupt immersion.

Why use Depth API?

The Scene Model enables the creation of room-scale, mixed reality experiences featuring realistic occlusion. However, it cannot handle occlusion for objects that are dynamically moving within the user’s view, such as hands, limbs, other people, and pets. To achieve realistic occlusion with these dynamic elements, you must also use the Depth API.

Use cases	Depth API	Scene API
Static Occlusion: the occluding real-world environment remains immobile (static) throughout the lifetime of the app, i.e. no moving objects such as hands, people or chairs.	✔	✔
Dynamic Occlusion: the occluding real-world environment contains mobile (dynamic) elements, e.g. the users hands, other people, pets.	✔	✖
Raycasting: Computing intersection of a ray and real-world surfaces. Supports use cases like content placement.	✔	✔
Physics/Collisions: interactions between virtual content and the real-world, like a virtual ball bouncing off of a physical couch.	✖	✔

How do I start with Depth API?

Depth API is only supported on Quest 3 and newer devices.

Before you begin working with Depth API, familiarize yourself with Passthrough. Having Passthrough in your app is required for receiving environment depth.

Requirements

Unity 2022.3.15f1 or higher, or 2023.2 or higher (Unity 6+ is recommended)
Meta XR Core SDK (v67.0.0 or above) by using one of the following methods:
- You may get it from the Unity Asset Store.
- Import the com.meta.xr.sdk.core (v67.0.0 or above) package from Unity’s package manager. Follow the instructions on importing individual SDKs in Unity outlined in the official Meta documentation.
An XR provider plugin
- Unity OpenXR Meta plugin extension (com.unity.xr.meta-openxr@2.1.0) (required for occlusion support in Unity versions 6+ and Meta XR SDK v74+)
- Oculus XR Plugin 4.2.0+ (Unity versions < 6 and SDK versions < v74)

Starting from version v67, we have updated the mechanism for retrieving depth textures, resulting in improved quality and performance. Although the Depth API is also supported in earlier versions, we recommend upgrading to version v67 for the best experience. For support with updating from previous versions, use the documentation in this github repository. For pre-v67 support, consult this older version of the documentation.

The Occlusions Overview section describes the most common use case of the Depth API and will later go through step-by-step implementation details.

Did you find this page helpful?