Audio in Virtual Reality

Audio is crucial for creating a persuasive VR experience. Because of the key role that audio cues play in our cognitive perception of existing in space, any effort that development teams devote to getting it right will pay off in spades, as it will contribute powerfully to the user’s sense of immersion. This is as true for small- or mid-sized teams as it is for design houses — perhaps even more so.

History

Audio has been a crucial part of the computer and video gaming experience since the advent of the first coin-op games, which filled arcades with bleeps, bloops, and digital explosions. Over time, the state of computer audio has steadily improved, from simple wave generators (SID, 1983) to FM synthesis (AdLib, 1987), evolving on to 8-bit mono samples (Amiga OCS, 1985; SoundBlaster, 1989) and 16-bit stereo samples (SoundBlaster Pro), culminating in today’s 5.1 surround sound systems on modern gaming consoles (Xbox, 2001).

Since the development of 5.1 surround, little has changed. The fundamental technology of playing waveforms over speakers is the same, and the game playing environment is still primarily the living room or den with a large television and speakers.

Enter VR

Virtual reality, however, is changing all this. Instead of a large environment with speakers, virtual reality brings the experience in close to the player via a head-mounted display (HMD) and headphones. The ability to track the user’s head orientation and position significantly empowers audio technology.

Until now, the emphasis has typically been placed on the visual aspects of virtual reality (resolution, latency, tracking), but audio is also important to provide the greatest sense of presence possible.

This document discusses the challenges, opportunities, and solutions related to audio in VR, and how some of the techniques learned in traditional game development must be revisited and modified for VR. It is not intended to be a rigorous scientific study of the nature of acoustics, hearing and human auditory perception. Its intended audience includes anyone with an interest in audio and VR, including sound designers, artists, and programmers.

For More Information

If you are interested in learning more, see the glossary for important terms, and review the rest of the topics in this section.

Start Using Oculus Tools

To get started with Oculus audio tools, see the Oculus Audio SDK Guide.