TL;DR:Oculus is releasing a new technology aimed at reducing system hardware requirements while maintaining content quality across a wider array of hardware. Asynchronous Spacewarp (ASW) is a frame-rate smoothing technique that almost halves the CPU/GPU time required to produce nearly the same output from the same content. Like Asynchronous Timewarp (ATW), ASW is automatic and enabled without any additional effort from developers.
VR Is Demanding
Rendering convincing, life-like virtual reality environments at a rate fast enough to create a sense of presence is demanding. VR hardware needs to be compact, power efficient, but at the same time very capable. VR software demands near real-time latency from every aspect of the system and the application. Developers need to remain mindful of all these elements to produce an experience that's immersive and enjoyable. Given all this, it's amazing that we've come as far as we have in such a short time.
At Oculus, we've introduced several technologies such as Direct Mode and Asynchronous Timewarp that automatically help developers minimize latency and improve the experience under disparate software and hardware configurations. Today, we are releasing a new technology which automatically scales existing content to more affordable hardware.
Let's Do the Timewarp Again
Early DK2 prototypes first demonstrated basic Timewarp (TW). This was an orientation-only, head-rotation limited reprojection of frames to the lowest possible latency. It was synchronous with rendering but demonstrated the effectiveness of reducing latency by using head movement to predict where images would visibly land.
Asynchronous Timewarp was introduced at Rift launch and provided an automatic, asynchronous version of time warp. It works by intervening every frame to time warp the last submitted eye buffers, even if the application is taking too long rendering the next frame. More about ATW can be found in these two blog posts:
ASW builds on top of the virtual reality smoothing experience of ATW. ATW ensures that the experience tracks the user's head rotation. This means an image is always displayed in the correct location within the headset. Without ATW, when a VR application misses a frame, the whole world drags—much like slow-motion video playback. Encountering this while in VR is extremely jarring and generally breaks presence. ASW goes beyond this and tracks animation and movement within the scene to smooth over the whole experience.
What Is this Sorcery?
ASW generates extrapolated frames from previous frames generated by the VR application. On the surface, this sounds quite a bit like ATW, which is capable of extrapolating for only a single thing: the user's head rotation. While this covers many of the hitches, it's not a magic bullet. ASW works in tandem with ATW to cover all visual motion within the virtual reality experience. This includes character movement, camera movement, Touch controller movement, and the player's own positional movement. If the application falls behind the display's frame rate, the experience typically remains smooth and enjoyable.
An example of this extrapolation is shown below. We have a scene in which a held gun which is moving from right to left in frames 0 and 1 (generated 1/45th of a second apart), and we want to generate an extrapolated frame from that movement (1/90th of a second after frame 1). We detect the movement of the gun and generate a new frame which we display on behalf the application.
When it comes to virtual reality, ATW and ASW are siblings and complement each other. Timewarp is great for accommodating head rotation. In fact, it's perfect for static images at a distance. For applications like 360 videos and features on the horizon, Spacewarp is unnecessary. Conversely, Spacewarp is pretty good for animated objects up close but not so great at tracking head rotation.
With ASW, Oculus is building on ATW to produce the best virtual reality experience possible. Effective latency is kept low and head tracking as before is smooth—and now, moving elements within VR are also kept smooth.
What's the Catch?
Just as with ATW, ASW is active and enabled for all applications without any developer effort.
There's no completely free lunch, however. ASW doesn't scale well below half the display's refresh rate. Depending on what's being displayed, there may be visual artifacts present as a result of imperfect extrapolation. Typical examples include:
Outside of point 4, you shouldn't avoid scenarios that produce these artifacts but rather be mindful of their appearance. ASW works well under most, but not all, circumstances to cover sub-90fps rendering. We feel the experience of ASW is a significant improvement to judder and is largely indistinguishable from full-rate rendering in most circumstances.
As a Developer, What Can I Do to Make ASW Work Best?
ASW enables a class of computers that were previously unable to drive VR. This means on recommended specification systems ASW should rarely be seen, if at all. Developers should maintain 90 fps rendering on recommended spec systems. Without any additional effort from the developer, the experience generally will scale to minimum spec machines and use ASW as needed. This is because apps that run at 90 fps on recommended spec systems can typically run at 45 fps on min spec systems. ASW and VR in general will benefit from the following suggestions:
Do I Need a Supercomputer for This?
The more capable the machine is, the less likely it is to need ASW. ASW and its scene extrapolation is activated only when the application cannot maintain the nominal frame rate of the VR display. This means if you built a top-of-the-line machine, then you may never see ASW in action.
The hardware requirements for ASW are modest. This functionality has been enabled on all current-generation AMD GPUs (RX 400 series) and previous- or current-generation Nvidia GPUs (GTX 900 or 1000 series).
Windows 8 or later is required for ASW, but the latest version of Windows 10 provides the best support. We developed this technology with assistance from Microsoft, AMD and Nvidia—special thanks to everyone there for their support.
Coming Soon to a System Near You
ASW is available with the release of the Oculus 1.10 runtime. It will be enabled across all ranges of hardware and systems that support the feature, and activated for all applications. ASW will automatically engage whenever the application needs extra time for rendering. For developers the Oculus Debug Tool will provide support for controlling ASW for development purposes.