Bringing Phase Sync to Mobile VR

Oculus Developer Blog
|
Posted by Jian Zhang, Loren McQuade, Neel Bedekar
|
December 7, 2020
|
Share

Phase Sync is a frame timing management technology that we first introduced in the Oculus PC SDK, which can help minimize pose latency by dynamically adjusting frame timing. We are excited to announce that Phase Sync is now available on the Oculus Mobile SDK and has been integrated into UE4 and Unity.


Phase Sync on Mobile VR

Unlike the Oculus PC SDK, the Oculus Mobile SDK has been using fixed-latency mode to manage frame timing since its inception. The philosophy behind fixed-latency mode is to finish everything as early as possible to avoid stale frames. It achieves this goal well, but with our release of Quest 2, which has significantly more CPU and GPU compute than our original Quest, a lot of apps can finish rendering their frames earlier than planned. As a result, we tend to see more “early frames”' in Quest 2’s VrApi log line. The SDK does already have some ability to control latency under the current fixed-latency mode; e.g. extra latency mode / non-extra latency mode settings. However, these are not flexible enough for developers to handle a dynamic workload, which means that if developers disable extra-latency mode but the app takes longer than expected to finish, the app generates a lot of stale frames, which negatively impacts the user experience since it can cause frame judder.

Compared with fixed-latency mode, Phase Sync handles frame timing adaptively according to the app's workload. The aim is to have the frame finish rendering right before our compositor needs the completed frame, so it can save as much latency as possible, and also not missing any frames. The difference between Phase Sync and fixed-latency mode can be illustrated in the following graph on a typical multi-threaded VR app.

We can see that Frame N’s complete time is a lot closer to its compositing time under Phase Sync, which is the source of Phase Sync’s latency savings. The above is just a simplified illustration of how Phase Sync works, and more technical details can be found in our Oculus Connect 4 Phase Sync talk.

How to enable Phase Sync On Quest / Quest 2

Phase Sync is available to developers with our V23 Mobile SDK as an opt-in feature, and we have also integrated it into the latest UE4 and Unity versions for developers to enable it easily.

For UE4 developers

To enable it in UE4, download and build UE4.25 from our github, and check the “Phase Sync” check box under Project Settings->OculusVR.

For Unity developers

To use Phase Sync in Unity, you must be using the Oculus XR Plugin, under the XR Plugin Management system. Phase Sync is supported on Unity version 2019.4 and beyond, and inside Unity, the Oculus XR Plugin version 1.7.0 and beyond. You can select the following checkbox to enable it in the Oculus-specific settings:

Once you have successfully enabled Phase Sync in your app, you can confirm that it works by running“adb logcat -s VrApi”. There should be a component labeled “Lat=-1”, which means that the latency is managed dynamically by Phase Sync. In the fixed-latency mode, the value would be “Lat=1” ( extra latency mode ) or “Lat=0” ( non-extra latency mode ) under non-phase sync.

You can also read the “Prd=?” value to compare your application’s latency difference before and after enabling Phase Sync. Prd is a direct measure of the prediction latency that your app incurs, so under phase sync, this value will be lower.

For native developers

We don’t plan to release a public API for this feature inside VrApi. Instead, Phase Sync will be the default Vr Timing management method in our OpenXR implementation. Since we have rolled out our formal OpenXR support, we encourage native developers to switch over and test the feature.

Impacts and considerations

We have enabled Phase Sync on a couple of Oculus apps, for which the latency savings have been fairly impressive. For example, we were able to achieve a 10 milliseconds latency reduction in Oculus Home with Quest 2, and an 8 milliseconds reduction with Quest.

We encourage every in-development app to enable it, especially if your app is latency sensitive (if it uses hand tracking, for example). Below are a few additional tips to help inform your decisions.

  • No Overhead: Phase Sync doesn’t have a performance overhead, so it shouldn’t affect your CPU / GPU usage in any significant way.

  • Workload spikes: Phase Sync has an insurance padding to avoid pushing the frame too close to compositing time, so it can handle some amount of variance in frame workload. However, if the app’s workload fluctuates violently or spikes a lot, it might trigger more stale frames than non-Phase Sync mode.

  • Low FPS behavior: When an app can’t make framerate, stale frames will be triggered inevitably. Non-Phase Sync mode will just let it be, but Phase Sync will try to reduce stale frame numbers to make at least some frames display at the correct time, so you should be able to see a lower stale frame count under this situation. However, FPS might be slightly lower and the Prd value could be higher. The rule of thumb is: if a frame is stale, which means the frame was predicted at the wrong time, the value of Prd isn’t important anymore.

  • Working with Late-Latching: Last year, we shared our UE4 late-latching solution, which is another latency reduction technology. We are happy to say that Late-Latching and Phase Sync can work together quite well. On a certain scale, they are complementary to each other, because Phase Sync can save more latency when the app is running fast and Late-Latching can help more if the app has an expensive rendering thread. If you enable both, it can almost guarantee some latency reduction.

  • Extra latency mode: Once Phase Sync is enabled, the extra latency mode setting will be ignored.

  • CPU&GPU metric: In our VrApi Logcat, we have one metric called “CPU&GPU”. Under Phase Sync, we measure it from waitFrame returning ( main thread wake up ) to Gpu finishing, the value is likely bigger than non Phase Sync mode, which is expected.

Please let us know if you see any abnormal app behaviors when you enable Phase Sync in your app.


Reference Reading

There are quite a few concepts in this post that we didn’t give too much detail about, as they have been covered in our previous blog posts.

For Rift Phase Sync: Oculus Connect 4 | Deep Dive into the Oculus Rift SDK

For extra latency: Understanding Gameplay Latency for Oculus Quest, Oculus Go and Gear VR

For VrApi log: OVR Metrics Tool + VrApi: What Do These Metrics Mean?

For late latching: UE4 Late Latching for Oculus Quest: Design, Benefits and How to Get Started