Composite the Scene using OBS

Important

All Oculus Quest developers MUST PASS the concept review prior to gaining publishing access to the Quest Store and additional resources. Submit a concept document for review as early in your Quest application development cycle as possible. For additional information and context, please see Submitting Your App to the Oculus Quest Store.


We're no longer accepting submission of 32-bit Oculus Quest apps. Any new or updated Oculus Quest application needs to be 64-bit. Please contact Oculus if you are unable to comply with this policy. Oculus Go and Gear VR apps will not be affected by this change.

This page describes the process of configuring OBS Studio using the Oculus plugin provided in the tools download to compose the scene.

Install the OBS Plugin

If you have not done so already, install OBS Studio (x64, Windows) now. Then, navigate to the root folder of your OBS install, obs-studio. Copy the entire contents under oculus-mrc_OBS\versionXX folder from the Mixed Reality Capture Tools to the obs-studio root. You only need to do this once.

  • This OBS plugin is compatible with both OBS Studio 23.2.1 and 24.0.x. Please copy the correct version of plugin corresponding to the OBS Studio version you use. You may have to recopy the OBS plugin after upgrading OBS Studio from 23 to 24.

Composition Setup

Before you start compositing your mixed reality scene, make sure you’ve completed the Camera Calibration process. The process described below may require additional configuration or customization based on your camera and computer. The OBS Wiki describes how to use the software in detail.

  1. Create your new scene and add the “Oculus MRC” source. Right-click the source and select Group Selected Items. Rename this group “fg”.
  2. Right-click the “fg” group and select Copy.
  3. Right-click on an empty space in the “Sources” window and select Paste (Duplicate). Rename this group “bg”.
  4. Add a “Video Capture Device” to the middle of the “fg” and “bg” groups. You can configure it to use the same camera device in calibration
  5. Crop the “fg” and “bg” groups individually -
    • Right-click each group and add a “Crop / Pad” filter in each
    • Uncheck “Relative” checkbox
    • For “fg” group, set (X/Y/Width/Height) to (1920, 0, 1920, 1080)
    • For “bg” group, set (X/Y/Width/Height) to (0, 0, 1920, 1080)
  6. Configure the “Video Capture Device” by adding the Chroma Key and Render Delay filters. These values will depend on your setup and configuration.
  1. (optional) If the composited scene looks dirty (i.e. visible lines on the foreground), add the following items to your scene:
    • Add a “Color Source” into your “fg” group, put it below the “Oculus MRC” source. Set the color to something that does not appear in the content (Green #00ff00 is usually a good choice). Set the size to (3840, 1080).
    • Add a Chroma Key filter to your “fg” group. Tweak the parameters until you see a clean composition.

Begin Capture

Start the application in your Oculus Quest that you want to capture in mixed reality. Then -

  1. Double-click either of the “Oculus MRC” sources. In the properties window that opens, enter the IP Address that you recorded in the Camera Calibration process. Click Connect.
  2. Adjust the Render Delay so that the recorded real-world controller movement matches the in-app controller/hand movement.
  3. Adjust the Sync Offset in OBS’ Advanced Audio Properties if the audio and video of the capture are out of sync.
  4. Record or stream your mixed reality experience.
  5. When finished, Disconnect in the “Oculus MRC” sources.

Troubleshooting

  • If you’re unable to connect your Oculus Quest to OBS. Verify that your PC and HMD are on the same network and that mrc.xml is present on the device, then restart your Oculus Quest.
    • You can verify MRC is activated properly with ‘adb logcat -s OVRPlugin’ and look for ‘[MediaManager]’ in the log. If you see ‘[MediaManager] MRC camera found’ and ‘[MediaManager] Listening at 0.0.0.0 tcp_port 28734’, it indicates the app is ready for the connection from OBS.
  • If you didn’t get a smooth video source in the OBS plugin, but the performance of the game looked fine under headset, that usually indicated the computer performance was the bottleneck. Please check Device and System Configuration for detailed information.
  • The mixed reality capture quality and latency is heavily depend on the quality of the wireless network connection. If you are seeing unstable performance or heavy video latency, please use a dedicate wireless router with 5G bands.
    • If your network is unable to handle the default setting, you can adjust the bitrate with ‘adb shell setprop debug.oculus.mrc.bitrate [videobitrate]’. The default bitrate is 8000000 if your camera is 1080p. You can set it to a smaller number and relaunch the app.