Passthrough API Overview
Health and Safety Recommendation: While building mixed reality experiences, we highly recommend that you evaluate your content to offer your users a comfortable and safe experience. Please refer to the
Health and Safety and
Design guidelines before designing and developing your app using Passthrough.
Passthrough provides a real-time and perceptually comfortable 3D visualization of the physical world in the Meta Quest headsets. The Passthrough API allows developers to integrate the passthrough visualization with their virtual experiences.
Note: If you are just getting started with this Meta XR feature, we recommend that you use
Building Blocks, a Unity extension for Meta XR SDKs, to quickly add features to your project.
Passthrough is rendered by a dedicated service into a separate layer. Because apps cannot access images or videos of a user’s physical environment, they create a special passthrough layer. The XR Compositor replaces this layer with the actual passthrough rendition.
By default, the Passthrough API creates this passthrough layer automatically (using automatic environment reconstruction), and submits it directly to the XR Compositor. Most of the time, you will use automatic environment reconstruction because you will not know the geometry of the user’s environment ahead of time.
You can override automatic environment reconstruction by using
Surface-Projected Passthrough. This lets you define the geometry onto which passthrough images are projected. This approach leads to a more stable rendition of Passthrough in cases where the app has already captured parts of the user’s environment.
Regardless of which passthrough layering you use, there are two ways to customize Passthrough - composite layering and styling:
- Composite layering lets you specify the placement of the passthrough layer relative to virtual content (overlay, underlay) in the XR Compositor stack and how it should blend with the virtual content. By using alpha masking, you can specify where on the screen Passthrough appears.
- To achieve more realistic blending of passthrough and virtual elements based on their depth see Depth API
is available for you to review.
- Styling lets you colorize the passthrough feed, highlight edges, and perform image processing effects such as contrast adjustment and posterizations of the image, by stair-stepping the grayscale values to achieve the desired effect.