Mixed Reality Capture Setup Guide - Oculus Quest

Mixed reality capture places real-world objects in VR. In other words, it mixes images from the real world with the virtual one.

Oculus mixed reality capture is designed for developers to share an intimate VR experience with users. By placing yourself and others in a VR scene, you can demonstrate the interaction between the user and virtual world.

Mixed reality capture is in development. This setup process may change.

How does mixed reality capture work?

Mixed reality capture uses an external camera to capture real-world images, against a green-screen, and combine them with a scene generated by a VR application to create a single blended scene. A user can then interact with the VR scene, and a video can be created that shows the user’s interactions in the real-world overlaid on the VR experience

Before You Begin

Before any calibrations, configuration, or implementation, you need to acquire a PC and camera that meet the performance requirements of mixed reality capture. Review the Device and System Configuration page for hardware and device specifications required for Mixed Reality Capture.

You’ll need both an Oculus Quest and Oculus Rift or Rift S (Rift only required to configure and perform setup).

Review the guidelines for creating a safe use environment in the Health and Safety warnings located at https://www.oculus.com/legal/health-and-safety-warnings/.

You’ll also need an app that supports Mixed Reality Capture. Currently only Unity Apps built on a certain version of the Oculus Integration support Mixed Reality Capture. See the Mixed Reality Capture page for more information.

Finally, download the MRC Tools for Oculus Quest, you’ll need various components throughout the setup process.

Getting Started

There are two steps to capturing mixed reality capture scenes. Setting up your Oculus Quest and external camera, and compositing the final scene.

Setup

First, you’ll need to configure your Oculus Quest for Mixed Reality capture by installing the provided app via ADB.

Then, you’ll calibrate your external camera to capture the real world images. The calibration process removes image distortion and determines the depth and position of objects captured in the scene. Information about calibrating the camera is found on the Camera Calibration page.

Compositing

Once you’ve finished the setup and configuration, you can launch the mixed reality capture capable app and composite the final scene together in OBSusing the Oculus Mixed Reality plugin. If you have not previously installed OBS, please do so now. Please review the Composite the Scene page for information about this process.

Non-Oculus equipment/service references are not recommendations, guarantees, or endorsements of products or performance. Your results may vary. These are instead options to vary your own customized experiences. Furthermore, incorrect installation or use of any products or items can damage your computer, space, or otherwise impact performance. If you’re not experienced or qualified to do so yourself, please seek assistance.