Mixed Reality Capture

Important

All Oculus Quest developers MUST PASS the concept review prior to gaining publishing access to the Quest Store and additional resources. Submit a concept document for review as early in your Quest application development cycle as possible. For additional information and context, please see Submitting Your App to the Oculus Quest Store.

Mixed reality capture for Oculus Quest places real-world people and objects in VR. This guide will review how to configure support for mixed reality capture in your Oculus Quest app.

This feature is in development. Implementation and functionality may change.

Introduction

Mixed reality capture places real-world video footage of a user to be composited with the output from a game to create combined video that shows the player in a virtual scene.

Information about device and camera setup can be found in the Quest Mixed Reality Capture Setup Guide.

Once an application is built to use mixed reality capture, users can launch applications in Mixed Reality mode and use the Oculus MRC Plugin for OBS.

Preparation

Mixed reality capture may be used by any application built on the Oculus Integration 1.38, or later.

Mixed Reality Performance

Rendering from the additional 3rd-person camera will impact the application performance. The CPU/GPU frequency of the Oculus Quest would be boosted temporarily to provide the extra performance when mixed reality capture is activated.

The video stream sent to OBS is encoded and transmitted every other frame, so the streamed video will be at 1/2 the framerate of the application running on the Oculus Quest. For a quality capture, ensure that the application runs at 60 fps or higher on the Oculus Quest when mixed reality capture is activated.

Compositing the Scene

Mixed reality capture for Oculus Quest supports external composition using the Oculus-provided OBS plugin. The Oculus Quest will stream the foreground and background scenes to a PC that is capturing the real-world image. The OBS plugin will combine the scenes together for the final Mixed Reality recording. Information about using the OBS plugin can be found in the Quest Mixed Reality Capture Setup Guide.

Sample Scene

A sample map with mixed reality capture enabled is available in the Oculus Unreal GitHub repository (access instructions here) in Samples/Oculus/MixedRealitySample.