Discontinuing Mobile SDK support means:

For more information about this deprecation, see Meta All In on OpenXR: Deprecates Proprietary APIs.

Asynchronous TimeWarp (ATW) transforms stereoscopic images based on the latest head-tracking information to significantly reduce the motion-to-photon delay, reducing latency and judder in VR applications.

In a basic VR game loop, the following occurs:

The following shows a basic example of a game loop:

Basic Game Loop

When frame rate is maintained, the experience feels real and is enjoyable. When it doesn’t happen in time, the previous frame is shown which can be disorienting. The following graphic shows an example of judder during the basic game loop:

Basic Game Loop with Judder

When you move your head and the world doesn’t keep up, this can be jarring and break immersion.

ATW is a technique that shifts the rendered image slightly to adjust for changes in head movement. Although the image is modified, your head does not move much, so the change is slight.

Additionally, to smooth issues with the user’s computer, game design, or the operating system, ATW can help fix irregularities or moments when the frame rate unexpectedly drops.

The following graphic shows an example of frame drops when ATW is applied:

Game Loop with ATW

At the refresh interval, the Compositor applies ATW to the last rendered frame. As a result, a frame with ATW will always be shown to the user, regardless of frame rate. If the frame rate is very low, flicker will be noticeable at the periphery of the display. But, the image will still be stable.

ATW is automatically applied by the Compositor; you do not need to enable or tune it. Although ATW reduces latency, make sure that your application or experience makes frame rate.

Stereoscopic eye views are rendered to textures, which are then warped onto the display to correct for the distortion caused by the wide angle lenses in the headset.

To reduce the motion-to-photon delay, updated orientation information is retrieved for the headset just before drawing the time warp, and a transformation matrix is calculated that warps eye textures from where they were at the time they were rendered to where they should be at the time they are displayed. The warped pixels are almost exactly correct. A sharp rotation will leave some pixels black at the edges, but this turns out to be minimally distracting.

The time warp is taken a step farther by making it an “interpolated time warp.” Because the video is scanned out at a rate of about 120 scan lines a millisecond, scan lines farther to the right have a greater latency than lines to the left. On a sluggish LCD this doesn’t really matter, but on a crisp switching OLED, users may feel like the world is subtly stretching or shearing when they turn quickly. This is corrected by predicting the head attitude at the beginning of each eye, a prediction of < 8 milliseconds, and the end of each eye, < 16 milliseconds. These predictions are used to calculate time warp transformations, and the warp is interpolated between these two values for each scan line drawn.

The time warp may be implemented on the GPU by rendering a full screen quad with a fragment program that calculates warped texture coordinates to sample the eye textures. However, for improved performance the time warp renders a uniformly tessellated grid of triangles over the whole screen where the texture coordinates are setup to sample the eye textures. Rendering a grid of triangles with warped texture coordinates basically results in a piecewise linear approximation of the time warp.

If the time warp runs asynchronous to the stereoscopic rendering, then it may also be used to increase the perceived frame rate and to smooth out inconsistent frame rates. By default, the time warp currently runs asynchronously for both native and Unity applications.