This guide describes best practices for the Gear VR Controller, an orientation-tracked controller for use with Gear VR.
The Gear VR Controller provides the following inputs:
Our SDK provides a shoulder/arm/wrist model to estimate controller position. The result of this calculation is returned as the controller’s position. This is based off the rotation of the controller as well as an estimate of the current head rotation in relation to the body of the user. You can see the source of this calculation in the Mobile SDK, in the VrController sample project.
Our Unity and Unreal integrations include a model of the controller for use in your game. There is no requirement to use this as is, so feel free to customize it to best fit your application.
The Gear VR Controller works extremely well as a pointing device. If you have used the controller in Home or the Universal Menu, you know how intuitive it is to point and select with the controller.
For interacting with UI, we recommend rendering the controller in the scene, drawing a laser shooting out of the tip (either extending all the way to the UI, or fading after a few inches), and rendering a cursor over your UI at the intersection of the ray cast from the controller. We strongly recommend using our arm model to position the controller unless you have a good reason not to, as this position will match what people are used to in Home, the Universal Menu, and most other apps. If you showing a controller doesn't make sense in your app but you want the controller rotation to control your UI cursor, make sure your ray cast origin is your camera position.
If you have a menu that can be paged through, or smooth scrolled through, use the controller touchpad. Swiping across the touchpad should move your UI in the same direction as the swipe. For example, if you have a long vertical page, swiping up should move the page upwards to let you see more at the bottom of the page. See Home for a reference implementation of this interaction. It is generally best practice to allow the user to select with either the trigger or touchpad click, unless you need to reserve one of those for some other action.
If you are using the controller to aim at objects at multiple depths (e.g., for a shooter), we strongly recommend that you leave the full laser pointer on at all times. When your ray cast origin is not the same as the camera origin, your cursor may end up behind objects in the scene, which is confusing. In that case you might obscure the cursor, or render it through the object, causing depth issues. However, if the laser is visible the player can easily figure out which way they need to move the controller to aim at the intended object.
A common game interaction scenario is looking over a table surface or ground plane from above, and interacting with items on the surface of the table or plane (e.g., a board game or strategy game). It may be more visually impressive to move in closer and view the surface at a level angle, but this makes pointing at further objects more difficult. While zoomed in, a much smaller angle of vertical rotation will move the cursor over a much larger distance on the surface than if you are much further away from the surface. Also, if you have various objects on the table, your ray cast is much more likely to be occluded the closer to the surface you are. Make sure that you can interact with every part of your play space with ease, and consider allowing the player to change vantage points so they can always reach the far corners of the game board.
To make selections of 3D objects on a surface, it may make sense to pass your ray cast through the objects, and instead only intersect with the surface. If a user looks over a chess board at an angle and wishes to select their pawn, it may be difficult to point at the pawn if the queen is standing in front of it. In this case, the user would probably prefer to point to the board square the pawn is standing on instead of at the piece itself.
This would be a preferable interaction if some objects are likely to occlude objects behind them. It is a good idea to always highlight the object you are selecting. It is also probably best to make selection slightly 'sticky', such that it is harder to deselect something than it is to select (e.g., the hit box becomes slightly larger when you are over it). This can reduce frustration if the player accidentally twitches slightly when pulling the trigger.
There are a few ways to use the controller for driving. One option is to have the user tilt the controller on its side and grip the bottom with one hand and the top with the other. Then the player can steer using the controller like a steering wheel. The disadvantage of this approach is that it limits the player’s ability to hit the trigger. It is still possible but it may not be comfortable to do so, so avoid using that as a primary mechanic. Also, note that the touchpad is still reachable, but will be rotated. If you do use this control scheme, we recommend allowing the player to select whether the touchpad end is in their left or right hand. If you use the touchpad, be sure to take this into account.
Another way to control a vehicle is the one-handed tilt approach: tilt left to steer left, tilt right to steer right. This makes trigger and touchpad use easier. You may also consider not using controller rotation at all, and simply using touchpad position. We report touchpad as a value between -1 and 1 for the x and y axis, so it should be simple to use this value in the same way you use a controller thumbstick. However, note that thumbsticks are self-centering, while touchpads are not. When the player releases the touchpad, it will not return to 0,0, and because there is no tactile feedback of when the thumb is centered, you should probably specify a large 'dead zone' at the center of the pad that counts as 0,0.
The controller makes an excellent flight stick, though the lack of physical constraints on rotation can cause some interesting cases. We have covered this topic in detail in our Developer Blog post Aircraft Controls using the Gear VR Controller.
If you need the player to move around in your scene, the most comfortable option is teleportation. One option that works well is to include fixed waypoints that the player simply needs to point at and select. The advantage of this is it allows you fine control over the objects that are visible to the player, which lets you disable a lot of geometry for improved performance.
Some people find waypoint teleportation limiting and prefer free range movement. One scheme using the controller that works well is to use the capabilities of the touchpad in a manner similar to Robo Recall on Rift. When the player touches the touchpad without clicking, a laser is cast out of the controller (an arc may be preferred). Wherever the beam intersects with the ground show an indicator that points in a direction that will be the player’s forward after teleporting. You can then use the touch position on the touchpad to determine this rotation. Because the range for x and y are -1 to 1, simply use the arctangent of the x and y to determine the angle of rotation. Then, when the player presses the touchpad, they are teleported to the location indicated.
Even though the Gear VR Controller provides only the trigger and touchpad click, it is possible to do a great deal with the controller. The touchpad lets you interact in a variety of ways. The Gear VR touchpad is generally limited to tap and swipe, which are available on the controller. But the touchpad has a highly accurate position and a click, so you have a world of new options.
You can treat the touchpad like a D-pad, where presses in the four cardinal directions give you four different button presses. You can also use the touchpad surface for one set of actions, and the depress the touchpad to open up a second set of actions. For example, a very useful interaction is to press the touchpad to open up a radial menu in game, then use the position of your thumb on the touchpad to select one of the wedges in your menu and release to select it.
There are an infinite number of ways you can use the controller in your game, and we encourage you to experiment. Try other apps and see how they use it. Have people pick up and try to play for the first time, and see if they can figure out the controls without explaining them. Most importantly, make sure your game is fun or your experience is engaging.
For more information, see: