Spectator Camera for Casting

Spectator Camera is extending Mixed Reality Capture (MRC) to the Oculus app casting. It doesn’t require OBS setup used in MRC, and allows user to simply cast to cell phone. While casting and viewing first-person view from Oculus Quest, you can creatively switch between first-person view and third camera in the scene. We believe this will produce more engaging spectating experience with your friends, family, and community. Therefore, a starter sample below demonstrates how to integrate it for Oculus Quest apps.

Spectator Camera Sample

The Oculus Integration SDK provides a sample implementation. To open the scene, go to the Oculus > VR > Scenes > SpectatorMode.unity. Oculus Quest controller can be used to control drone-like camera movement.

  • Index trigger to switch view during casting.
  • Right thumbstick to move vertically and horizontally.
  • Left thumbstick to move forward and backward.

Use the SetPlatformInitialized and SetPlatformCameraMode APIs to enable the spectator camera. SetPlatformInitialized initializes the necessary platform library to leverage on casting. During casting session, SetPlatformCameraMode helps switch camera between first-person view and third-person spectator camera view.

    // Start is called before the first frame update
    void Awake()
    {
#if OVR_ANDROID_MRC
        OVRPlugin.Media.SetPlatformInitialized();
#endif
    }

    private void UpdateSpectatorCameraStatus()
    {

#if OVR_ANDROID_MRC

        // Trigger to switch between 1st person and spectator mode during casting to phone
        if (OVRInput.GetDown(OVRInput.Button.PrimaryIndexTrigger) || OVRInput.GetDown(OVRInput.Button.SecondaryIndexTrigger))
        {
            camMode = OVRPlugin.Media.GetPlatformCameraMode();

            if (camMode == OVRPlugin.Media.PlatformCameraMode.Disabled && readyToSwitch)
            {
                OVRPlugin.Media.SetPlatformCameraMode(OVRPlugin.Media.PlatformCameraMode.Initialized);
                StartCoroutine(TimerCoroutine());
            }

            if (camMode == OVRPlugin.Media.PlatformCameraMode.Initialized && readyToSwitch)
            {
                OVRPlugin.Media.SetPlatformCameraMode(OVRPlugin.Media.PlatformCameraMode.Disabled);
                StartCoroutine(TimerCoroutine());
            }
        }
#endif

    }

    void OnApplicationPause()
    {
#if OVR_ANDROID_MRC
        OVRPlugin.Media.SetPlatformCameraMode(OVRPlugin.Media.PlatformCameraMode.Disabled);
#endif
    }

    void OnApplicationQuit()
    {
#if OVR_ANDROID_MRC
        OVRPlugin.Media.SetPlatformCameraMode(OVRPlugin.Media.PlatformCameraMode.Disabled);
#endif
    }

Spectator Camera Performance

Rendering from the additional third-person camera impacts the app performance. Similar to Mixed Reality Capture, Oculus Quest CPU/GPU frequency will be temporarily boosted, and the streamed video will be at half the framerate of the app running on Oculus Quest.

Known Issue

When you switch view, the casting video is flipped upside down. We are aware of this issue and the workaround to resolve this is by setting SetMrcFrameImageFlipped to false. Once this issue is resolved, we’ll announce it in our Release Notes.

        if (camMode == OVRPlugin.Media.PlatformCameraMode.Disabled && readyToSwitch)
        {
            OVRPlugin.Media.SetMrcFrameImageFlipped(false);
            OVRPlugin.Media.SetPlatformCameraMode(OVRPlugin.Media.PlatformCameraMode.Initialized);
            StartCoroutine(TimerCoroutine());
        }