Social Experiences For People In and Out of VR

With WebVR, developers can more easily create interactive 360 experiences that reach a large audience across VR, mobile, and the web. React VR—an open source framework built upon React Native—is a great way to start building WebVR content.

In addition to enabling broad distribution, WebVR can also connect people in VR with their friends out of VR—regardless of what device they’re using. In this post, we will introduce the different ways to represent users in VR who don’t have a headset and share some new use cases that this can enable.

At F8, we showcased this video produced by the team at Breach VR to demonstrate how players immersed in a VR Pictionary game can have meaningful interactions and shared experiences with friends accessing the same virtual space on their phones, laptops, and other devices.

Join from any Device
WebVR content can be enjoyed on mobile or PC using a modern browser that supports WebGL—and can be experienced in a VR headset like Rift, Gear VR, Daydream, and HTC Vive using a WebVR-enabled browser. This opens up the possibility of creating common 3D environments or shared 360 experiences that can be accessed by friends across a range of devices.

By synchronizing the device orientation across VR headsets and mobile phones, we can map each person to a corresponding avatar representation in VR. For those already immersed in VR, the experience between a friend joining from VR and one who is joining from their phone is very similar.

Just like today’s websites use responsive design to appear differently when viewed on varying sized devices, responsive WebVR allows for different modes of communication and interaction across different device types. In the VR Pictionary example, we defined three possible roles with varying degrees of agency for friends joining the experience:

Avatars and User Representations
Exactly how a person should appear to others in VR is still an open question. Currently, a variety of avatar styles are used in various applications—from abstract models and cartoon representations to live video feeds captured by a phone’s camera.

Ultimately, it’s up to the developer to decide what representation is the most appropriate to use based on the application being built.

The avatar style can also adapt based on the context. For example, a more abstract avatar or profile photo of the person might be appropriate for those who have not gone through the process of customizing an avatar.

Communication Between People
When microphone and audio output is available (which is often the case with most VR headsets, phones, and laptops), the primary mode of communication between people in VR is talking to one another.

Additionally, physical expression in VR greatly enhances the sense of social connection. This can be achieved between people both in and outside of VR simply by matching the avatar’s gaze direction to the tracked camera viewport of people on laptops, phones, tablets, and VR headset.

Physical expression is also an extremely effective way to amplify presence between people. Even for those without tracked hand controllers, we can achieve this by enabling a custom reactions menu, keyboard shortcuts, or even interpreting vocal cues to trigger various facial expressions, animations, and pre-defined body gestures to convey a multitude of emotive poses.

Multiple Use Cases
Connecting with a group of friends in virtual space, regardless of which device they’re using, opens up a broad range of potential applications.

The concept video we put together for F8 2017 featured a casual game of VR Pictionary. In this scenario, people using phones would guess what a player was drawing with the tracked Gear VR Controller, while participants and observers could chime in from Facebook News Feed or Messenger. Casual gaming is just one of the many areas where we can leverage the power of cross-platform social VR.

Interactive VR films, for instance, can be taken to the next level by letting you share the experience with one or more friends, reacting to and commenting on the narrative together as it unfolds, perhaps even cooperating in order to influence the outcome of the story.

Multi-user device-agnostic VR is also ideal for planning with friends and family, be it choosing an Airbnb rental for a big group reunion or finding a new house to buy on Redfin by touring multiple 360 listings. Whether you’re fully immersed in VR or previewing through a mobile phone, you can jump into a shared experience and see what your friends are up to.

Likewise, the potential for collaboration tools in general is increased by the possibility of having any combination of artists, designers, project managers, developers, etc., gathering in the same virtual space to brainstorm, but without the barrier of requiring everyone to have a VR headset.

The benefits extends to VR training and education applications, where an instructor with a headset and tracked controllers can vividly communicate with a mixed group of students participating online via laptops, tablets, and headsets. The same goes for guided tours of museums or under-construction projects.

Finally, asynchronous social interactions in VR can also enhance content by letting people share, discuss, and converse from multiple spatially-anchored hotspots within a 3D scene or 360 media. Avatar representations can be used to ‘play back’ how your friends engaged with the piece of content in addition to what their reactions and comments are.

These are just a few examples of possible use cases for enabling social interactions across different devices using React VR. When anybody can join the experience from whatever device they have available, the possibilities are virtually endless.

Interested in working with us to build the above examples or define new use cases? Send us an email at webvr@oculus.com.