Building with WebXR at the Presence Platform Hackathon

Meta’s Presence Platform Hackathon took place from April 25-27, 2023 across two locations: London, UK and Menlo Park, CA.

I was fortunate enough to attend the event at the Meta offices in London, being one of over 150 developers selected from across the world to compete in small teams with the goal of building innovative mixed reality experiences using Presence Platform capabilities and Meta Quest Pro.

Five of these developer teams—including mine—built WebXR apps that ran directly inside the Meta Quest Browser, without having to install anything. WebXR developers might have been in the minority at the event, but there was a fantastic sense of community between us. If you haven’t already, we encourage you to checkout the wonderful WebXR Discord community to connect with us and other passionate WebXR developers.

WebXR has somewhat of a reputation for being tech-heavy or harder to work with—but I think all of the teams proved that this is not the case anymore. High-quality experiences are possible, from both a technical and design standpoint.

Following the event, we wanted to celebrate the Presence Platform Hackathon entries made by our community. Let’s take a look at the WebXR teams and what we built…

Team Members: Clément Cordier, Ariel Cohen, and Felix Herbst

The Handmade Stories team created a MR puppet theater that lets players animate puppets and manipulate objects in real-time using Hand Tracking.

Kids can watch the puppet shows live on a tablet or computer, and can even take part in the scene—like helping grow carrots! The team performed three shows for kids in different countries during the event.

The pitch video demonstrates how traveling parents can use this experience to connect with their young children back home with playful stories, interaction and games.

Handmade Stories was built using Needle Engine, a web and WebXR runtime that allows creators with Unity or Blender background to develop within that environment and create experiences that run directly in the browser—both for WebXR and desktops/mobile devices.

Despite only being a team of three, Handmade Stories delivered an experience that delighted the judges, placing in the top three hacks in the London Event—a fantastic achievement!

You can try “Handmade Stories” here.

Team Members: James Kane, Florian Isikci, Yin-Chien Yeap, Vincent Trastour, and Terrence Reilly

Team Nano Nav, also hacking in London, worked on building an MR game inspired by the classic ‘90s games Lemmings and The Incredible Machine. In Nano Nav, a group of tiny robotic aliens arrive through a portal into the player’s environment while exploring our planet. The player’s task is to escort them past a series of hazards to a goal destination.

The team was particularly interested in the new and unique challenges MR presents for game design. One issue is dynamically adapting content to the player’s environment. Another challenge was figuring out how to give players a reasonable explanation for why the game’s story is unfolding within their local environment.

The team used generative AI to quickly narrate and tutorialize the game, training a custom voice model based on “cartoon professor” YouTube search results, and adding that to a bed of original background music composed on-site. The game’s script also features comic beats intended to surprise and delight the player, enhancing a sense of fun and adventure.

Nano Nav was built using Wonderland Engine. While technical issues prevented them from achieving all their goals within time allowed, the team feels they’ve discovered a winning concept within a deep design space and intend to continue development and debut Nano Nav at Augmented World Expo on June 1.

Check out their video presentation below.

Team Members: Dulce Baerga, Tavius Koktavy, Diego Marcos, and Gheric Speiginer

Over in Menlo Park, the WIP XR team set out to build an immersive coworking app that would support both colocated and remote coworkers in MR, allowing everyone to share their “work-in-progress” with each other in an embodied way.

Colocated coworkers can collaborate together in a shared MR space, viewing and manipulating virtual objects. Remote coworkers can also join the same session, allowing them to work together as if they’re in the same physical space.

The WIP XR team implemented several networked widgets, including miniature dioramas, 3D scans, and 3D UI components useful for prototyping and sharing ideas. The project was implemented using Ethereal Engine, and includes full-body avatars, with Inverse Kinematics for arms, legs and even fingers. They also provided multi-platform support to allow users without headsets to join in via laptop/mobile device, leveraging webcams to enable immersive video conferencing.

The WIP XR team plans to extend this app to include networked screen-share widgets, so coworkers can easily share their screens with one another within customized MR coworking spaces. They intend to release all of this as a set of open source productivity widgets for Ethereal Engine.

Check out the team’s video presentation below.

Team Members: HiuKim Yuen, Shaw Walters, Zack Marlow, and Anthony Burchell

What does the future of social media look like in a near-future world where everyone wears glasses? Skyline is an MR client built on the AT protocol and Bluesky framework, exploring a new modality for social media.

In Skyline, messages appear as butterflies, fluttering in the air around the user. The user can grab them, bring them closer, and even eat them! The butterflies utilize the new Meta Reality Accelerator Toolkit to give context for planes in the space, which are used to provide the butterflies surfaces to travel to and from randomly.

This project was implemented using React Three Fiber and can be found here. Check out the team’s video presentation below.

Team Members: Diarmid Mackenzie, Timikha Valentius, and Thomas Henchman

Finally, back in London, my team’s hack was SquatMaster. With a team of just three, we focused on a basic concept of some key elements and built SquatMaster using A-Frame.

Our goal was to broaden the appeal of MR-supported headsets to people who had little prior interest in VR or MR. We were looking for a real-world activity where MR could enhance participants’ experience or performance while lifting weights.

We picked barbell squats, as they are a foundational element of many resistance training programs, and one that we had a bit of experience with. Since the user’s head and the barbell move synchronously during the squatting movement, we were able to infer the movement of the bar from the movements of the headset.

The app guides the user through a calibration process, in which it learns their height, their squat depth, and the position of their safety bars. It then guides them through one set of five reps, providing real-time data on their speed of movement, range of movement, and rest time between reps.

The judges were intrigued by this alternative approach to XR Fitness, and SquatMaster placed in the top five hacks at the London event.

You can check out our submission video below and see our accompanying slide deck here.

I want to finish with a few observations that seemed evident at the Hackathon about WebXR and its place in the mixed reality ecosystem.

First, our community is really special. Even if we’ve never met in person before, a lot of WebXR developers know each other already (or know of each other).

Second, the variety of tools available for WebXR development is remarkable. Each of the five WebXR hacks used a different framework, each with its own particular advantages.

And there are other frameworks out there too, like BabylonJS and PlayCanvas, which I am sure the WebXR developer community will use at a future event.

Third, speed of iteration is a superpower of WebXR compared to native app development. It’s also especially valuable for fast-paced events like the Hackathon.

When developing for WebXR, code changes can be available for testing within the time it takes to switch from your editor to your headset. Native app developers have a very different experience.

When it comes to designing UX for VR or MR, it can be hard to anticipate exactly how designs and models will look and feel within the headset—you need to actually experience them! Fast iterations make an enormous difference in terms of your ability to tweak designs until they feel just right.

Fourth, the flexibility of WebXR is a massive bonus. Several of the WebXR hacks include multi-platform support, with users on laptops, tablets, or phones interacting with users wearing headsets. The browser as a platform allows for this kind of cross-platform experience more-or-less for free with WebXR—whereas for native apps, distributing to mobile and laptops in addition to headsets adds a significant challenge.

From a technology perspective, the difference between what can be achieved in WebXR versus a native app is marginal. There are a few Presence Platform capabilities—like Interaction SDK and Voice SDK—that aren’t yet available in WebXR, but these features aside, there was very little that was implemented as a native app at either hackathon that couldn’t have been implemented just as easily in WebXR. On the other hand, the multi-platform features used in several of the WebXR hacks would have been much harder to implement in native apps.

Given the advantages of using WebXR, it will be interesting to see whether the share of developers choosing to use WebXR increases at future events.

If you are curious about WebXR and creating experiences for it, please do check out the links to the hacks above. And be sure to join the WebXR community on the WebXR Discord Server—we look forward to seeing you there!