Passthrough sample is available on GitHub, and its use is explained in detail below. In addition, the following projects provide examples of passthrough used with other APIs:

Open the Navigation Menu by pressing the Menu button on your left controller. Once in the Navigation Menu, you can see if your device supports passthrough, the name and description of your current level, and a list of levels you can travel to. Aim your right-hand controller at the desired level and press the right trigger to travel to it. Scroll through the list with the right thumbstick.

A common use of passthrough is as a background, where you can place virtual objects (Passthrough AR). You can do this easily, starting from the PTBackground scene:

Here are relevant assets in this scene that you can use and modify to fit your needs:

Darkening your space and lighting it in key areas is another common use of passthrough. From shining a top-down spotlight on a table game or art piece to isolating into a “focus mode” by blacking out everything except your immediate area or shining a volumetric flashlight paired with dynamic lighting, you can achieve much with passthrough even without scene understanding.

In the PTLighting level, you can find an example of using passthrough to create a focus mode. The general idea behind lighting passthrough is:

Here are the relevant assets in this scene that you can use and modify to fit your needs:

You can draw a cone with one or multiple meshes. If you use multiple meshes, note the number of pixels drawn in every frame. Multiple overlapping, blended meshes will often result in many redrawn pixels. Pay careful attention to performance if you use large translucent meshes to simulate a volumetric look.

In the PTOpacity level, you can see an example of using passthrough as an overlay. In this sample, you use the slider widget (or the thumbstick on your right-hand controller) to adjust the opacity of the passthrough layer from 0% to 100%.

Here are relevant assets in this scene that you can use and modify to fit your needs:

The PTStyles level demonstrates some of the styling options available on passthrough layers. You can control the styling using the widget in the level:

Here are the relevant assets in this scene that you can use and modify to fit your needs:

Instead of specifying color effects via parameters (as demonstrated in the PTStyles level), you can define them as an array of 256 values that map each (grayscale) input value to a new color. The PTColorMap level provides an example of setting colors this way. With the in-game widget, you can modify the 256 color values and see the impact on the styling.

Here are the relevant assets in this scene that you can use and modify to fit your needs:

You can colorize passthrough using color LUTs. As demonstrated in the PTColorLut level, you can use two LUT types: Color LUT and Interpolated Color LUT. The second one will blend two LUTs (source and target) using the LUTWeight value.

Here are the relevant assets in this scene that you can use and modify to fit your needs:

Another way to colorize passthrough layers is to use color scale and offset. The PTColorScaleAndOffset level demonstrates this method and animates both of those settings. You can use the in-game widget to stop and edit the color scale and offset.

Here are the relevant assets in this scene that you can use and modify to fit your needs:

Caution:

The lighting and color in this sample may cause seizures in people with epilepsy or sensitivity to light and color.

The PTLayer_Placement level shows how the placement of passthrough layers can affect your scene. To change the order of passthrough layers, use the right trigger to switch the layer between the overlay and underlay or use the left trigger to toggle depth support.

Here are the relevant assets in this scene that you can use and modify to fit your needs:

The PTMaskedBrush and PTMasking levels demonstrate experiences where a passthrough layer is masked and only visible in certain screen areas. In the PTMaskedBrush sample, the controller can be used as a brush to draw a projected passthrough stroke by pressing the left and right triggers. PTMasking showcases a circle mesh with masked projected passthrough (using the mPokeAHoleMask material) that is attached to the controller, which allows you to see your hands (you can change the attached controller with the left/right trigger).

You can re-use and modify the mPokeAHoleMask.uasset material to mask areas in your scene.

Here are the relevant assets in this scene that you can use and modify to fit your needs:

In the PTSurfaceProjected level, you can see an example of passthrough projected onto a mesh surface. Instead of mapping passthrough to a mesh reconstructed from stereo imagery, this takes an input static mesh and uses the vertex positions to calculate exact stereo mapping. Use this in cases where you have high confidence in real items matching virtual items, such as flat surfaces. For example, the demo scene looks like a TV setup with a view of the real world. Depth accuracy is better with surface-projected passthrough, especially at longer distances where a reconstructed passthrough mesh is less reliable.

This feature is also working for procedural meshes as demonstrated in the PTProceduralMesh level.

Note: The passthrough will not follow mesh modification after it has been added as a Surface Geometry, if you have modified your procedural mesh and want the passthrough to follow you’ll need to remove and add again your mesh.

In the PTSurfaceProjected level, you can see an example of a passthrough projected onto a mesh surface. Instead of mapping passthrough to a mesh reconstructed from stereo imagery, this takes an input static mesh and uses the vertex positions to calculate exact stereo mapping. Use this in cases where you have high confidence in actual items matching virtual items, such as flat surfaces. For example, the demo scene looks like a TV set up with a view of the real world. Depth accuracy is better with surface-projected passthrough, especially at longer distances where a reconstructed passthrough mesh is less reliable.

This feature also works for procedural meshes, as the PTProceduralMesh level demonstrates.

Note: The passthrough will not follow mesh modification after you add it as a Surface Geometry. If you have modified your procedural mesh and want the passthrough to follow, you must remove and add your mesh again.