What Does the Gear VR Controller Provide?
The Gear VR Controller provides the following inputs:
The controller also provides volume up, volume down, and home buttons, but these are reserved by the system.
Our SDK provides a shoulder/arm/wrist model to estimate controller position. The result of this calculation is returned as the controller’s position. This is based off the rotation of the controller as well as an estimate of the current head rotation in relation to the body of the user. You can see the source of this calculation in the Mobile SDK, in the VrController sample project.
Our Unity and Unreal integrations include a model of the controller for use in your game. There is no requirement to use this as is, so feel free to customize it to best fit your application.
How Can I Use the Controller to...
Interact With UI
The Gear VR Controller works extremely well as a pointing device. If you have used the controller in Home or the Universal Menu, you know how intuitive it is to point and select with the controller.
For interacting with UI, we recommend rendering the controller in the scene, drawing a laser shooting out of the tip (you can either extend this all the way to the UI, or just have it fade after a few inches), and rendering a cursor over your UI at the intersection of the ray cast from the controller. We strongly recommend using our arm model to position the controller unless you have a very good reason not to, as this position will match what people are used to in Home, the Universal Menu, and most other apps. If you showing a controller doesn't make sense in your app but you want the controller rotation to control your UI cursor, make sure your ray cast origin is your camera position.
If you have a menu that can be paged through, or smooth scrolled through, you should use the controller touchpad. Swiping across the touchpad should move your UI in the same direction as the swipe. For example, if you have a long vertical page, swiping up should move the page upwards to let you see more at the bottom of the page. See Home for a reference implementation of this interaction. It is generally best practice to allow the user to select with either the trigger or touchpad click, unless you need to reserve one of those for some other action.
Aim at 3D Objects In My Scene
If you are using the controller to aim at objects at multiple depths (e.g., for a shooter), we strongly recommend that you leave the full laser pointer on at all times. When your ray cast origin is not the same as the camera origin, your cursor may end up behind objects in the scene, which is confusing. In that case you might obscure the cursor, or render it through the object, causing depth issues. However, if the laser is visible the player can easily figure out which way they need to move the controller to aim at the intended object.
A common game interaction scenario is looking over a table surface or ground plane from above, and interacting with items on the surface of the table or plane (e.g., a board game or strategy game). It may be more visually impressive to move in closer and view the surface at a level angle, but this makes pointing at further objects more difficult. While zoomed in, a much smaller angle of vertical rotation will move the cursor over a much larger distance on the surface than if you are much further away from the surface. Also, if you have various objects on the table, your ray cast is much more likely to be occluded the closer to the surface you are. Make sure that you can interact with every part of your play space with ease, and consider allowing the player to change vantage points so they can always reach the far corners of the game board.
If you need to make selections of 3D objects on a surface, it may make sense to have your ray cast pass through the objects, and instead only intersect with the surface. If I am looking over a chess board at an angle and I wish to select my pawn, it may be very difficult to point at the pawn if my queen is standing in front of it. In this case, I would want to point to the board square the pawn is standing on instead of at the piece itself. This would be a preferable interaction if some objects are likely to occlude objects behind them. It is a good idea to always highlight the object you are selecting. It is also probably to make selection slightly 'sticky', that is its harder to deselect something than it is to select (e.g., the hit box becomes slightly larger when you are over it). This makes it less frustrating if the player accidentally twitches slightly when pulling the trigger and would have moved the pointer over something else.
Control a Ground Vehicle (Car, Tank, Etc.)
There are a few different ways you can use the controller for driving. One option is to have the user tilt the controller on its side and grip the bottom with one hand and the top with the other. Then the player can steer using the controller like a steering wheel. The disadvantage of the position is it limits the player’s ability to hit the trigger. It is still possible but it may not be comfortable to do so, so avoid using that as a primary mechanic. Also note that the touchpad is still reachable, but will be rotated. If you allow this control scheme, it would be nice to allow the player to select whether the touchpad end is in their left or right hand, and if you use the touchpad, make sure to take this into account.
Another way to control a vehicle would be the one-handed tilt approach: tilt left to steer left, tilt right to steer right. This makes using trigger and touchpad easier. You could also consider not using controller rotation at all, and simply use touchpad position. We report touchpad as a value between -1 and 1 for the x and y axis, so it should be simple to use this value in the same way you use a controller thumbstick. However, note that thumbsticks are self—centering, while touchpads are not. When the player releases the touchpad, it will not return to 0,0, and because there is no tactile feedback of when the thumb is centered, you should probably specify a large 'dead zone' at the center of the pad that counts as 0,0.
Controlling an Aircraft (Plane, Spaceship, Etc.)
The controller makes an excellent flight stick, though the lack of physical constraints on rotation can cause some interesting cases. There is no definitive answer to this problem, but I’ve provided one potential solution.
Move Around My Scene
If you need the player to move around in your scene, the most comfortable option is teleportation. One option that works well is to include fixed waypoints that the player simply needs to point at and select. The advantage of this is it allows you fine control over the objects that are visible to the player, which lets you disable a lot of geometry for improved performance. However, some people find this limiting, and would prefer free range movement. One scheme using the controller that works well is to use the capabilities of the touchpad in a manner similar to Robo Recall on Rift. When the player touches the touchpad without clicking, a laser is cast out of the controller (an arc may be preferred). Wherever the beam intersects with the ground show an indicator that points in a direction that will be the player’s forward after teleporting. You can then use the touch position on the touchpad to determine this rotation. Because the range for x and y are -1 to 1, simply use the arctangent of the x and y to determine the angle of rotation. Then, when the player presses the touchpad, they are teleported to the location indicated.
Do More Things Than Are Possible With Only Two Buttons
Even though you really only have the trigger and touchpad click, you can do a lot more than you think with the controller. The touchpad lets you interact in a variety of ways. With the Gear VR touchpad, you pretty much only have tap and swipe, which are still available on the controller, but because the touchpad has a highly accurate position and a click, you have a world of new options. You can treat the touchpad like a D—pad, where presses in the four cardinal directions give you four different button presses. You can also use the touchpad surface for one set of actions, and the depress the touchpad to open up a second set of actions. For example, a very useful interaction is to press the touchpad to open up a radial menu in game, then use the position of your thumb on the touchpad to select one of the wedges in your menu and release to select it.
There are an infinite number of ways you can use the controller in your game, and we encourage you to experiment. Try other apps and see how they use it. Have people pick up and try to play for the first time, and see if they can figure out the controls without explaining them. Most importantly, make sure your game is fun or your experience is engaging.
— The Oculus Team