Tech Notes: Aircraft Controls Using the Gear VR Controller

With the upcoming release of the new Gear VR Controller, people will be able to use their hand to point, drag and drop, tilt, shoot, and more in Gear VR apps. We’re discovering some very interesting use cases for the new Gear VR Controller, like controlling aircraft and spacecraft. The usual input for an aircraft is in the form of yaw, pitch, and roll. Yaw is the left-right rotation of the aircraft, pitch is up-down tilt, and roll is the length-wise rotation where one wing goes up and the other goes down. Generally, real aircraft have more control over pitch and roll, which are controlled by the wing flaps, than yaw, which is controlled by the tail rudder.

Real aircraft are either controlled using a yoke (steering wheel thing) or a stick. Fans of flight simulators are very familiar with flight sticks or HOTAS (hands on throttle and stick), where they’re able tilt the stick to input pitch and roll—and sometimes twist the stick to input yaw. These mechanical sticks are able to read the axes from the physical device and send it to the software, which then apply the yaw, pitch, and roll adjustments to the aircraft.

Because the Gear VR Controller provides an orientation instead of a three-axis input, in order to use it as a flight stick we’ll have to do some math to convert a rotation into yaw, pitch, and roll. You might be tempted to just grab the Euler angles from the rotation and call it a day, but this will cause some issues. Because a rotation can be expressed as different sets of Euler angles, you may find the controls to be unpredictable, and if you tilt left, there may be a point where controls suddenly swing to the right or vice-versa. Generally you want to avoid this, so we’re going to use vector math to calculate the specific angles we want.

To start breaking down the problem, it makes sense to think of the current controller rotation as the result of the sum of three hinges. The first hinge goes forward and backward and controls the pitch. This value can be found by projecting the current controller forward onto the YZ plane, then determining the rotation of this vector around the X axis. Fortunately, Unity provides a function to get this projection: Vector3.ProjectOnPlane(). Once we have the projection, we can find the angle using the arctangent of y/z of this new vector. There is one gotcha however. If the controller forward is parallel to the X axis, the projection will be zero, which means you won’t be able to get an angle. To avoid this, if I detect that the projection is zero, I use either the controller right vector or left vector, depending on if the controller is turned left or right (whichever is facing “forward” if the controller is sitting flat on a surface). The one downside of using this method is it will cause gimbal lock with yaw. Arctangent will return a value between -90 and 90 degrees, while we want a value in the 360 range. Fortunately, Unity has a function Mathf.Atan2() which lets you pass in your y and z separately, and will handle divide by zero and determining a result in the full 360 degree range.

After calculating pitch, you can multiply your original rotation by the inverse of this value, which will place your forward vector into the XZ plane. You can then grab the roll rotation using Atan2(forward.x, forward.z). If you then multiply your rotation by the inverse of the roll rotation, your forward vector will now be Vector3(0,0,1) or the world forward direction.

Now let’s imagine our yaw component as determined by a twist of the stick. We’ve moved the stick to be in its upright position, so the only rotation remaining is this twist rotation. Determining it is as simple as getting our up vector from the rotation and grabbing the arctangent of x/y. In the case where the controller is pointed exactly 90 degrees right, the yaw will always be zero. Also, without the physical resistance of a flight stick, it’s very hard to tell at certain angles whether you’re twisting the stick at all. For that reason, it might make sense to drop the yaw component altogether, since you can fly a plane using only pitch and roll.

One downside to this method is that it only lets you get a roll angle of up to 90 degrees. Once you roll past that, it reads as a 180-degree pitch forward with a 180-degree yaw. To allow greater roll, you can detect any yaw over 90 degrees (or some higher threshold if you want to allow larger yaws) and assume that was a mistake. You can then reverse the 180-degree rotation and determine the larger roll value.

Below is an implementation of these controls—they take a quaternion and produce a yaw, pitch, and roll value between -1 and 1.

#define DRAW_DEBUG_LINES using System.Collections; using System.Collections.Generic; using UnityEngine; public static class FlightStickSimulation { private const float MaxAngle = 60f; private const float DeadZone = 0.2f; private const float PitchAngleOffset = 45f; public static void CalculateYawPitchRoll(Quaternion rotation, out float yaw, out float pitch, out float roll) { Vector3 forward = rotation * Vector3.forward; var pitchProj = Vector3.ProjectOnPlane(forward, Vector3.right); if (pitchProj.sqrMagnitude < 0.001f) { forward = rotation * (forward.x > 0 ? Vector3.left : Vector3.right); pitchProj = Vector3.ProjectOnPlane(forward, Vector3.right); } #if DRAW_DEBUG_LINES Debug.DrawLine(Vector3.zero, forward * 3, Color.blue); #endif float pitchAngle = Mathf.Atan2(pitchProj.y, pitchProj.z); // cancel out pitch rotation = Quaternion.Euler(pitchAngle * Mathf.Rad2Deg, 0, 0) * rotation; forward = rotation * Vector3.forward; #if DRAW_DEBUG_LINES Debug.DrawLine(Vector3.zero, forward * 3, Color.red); #endif var rollProj = Vector3.ProjectOnPlane(forward, Vector3.up); float rollAngle = Mathf.Atan2(rollProj.x, rollProj.z); // cancel out roll rotation = Quaternion.Euler(0, -rollAngle * Mathf.Rad2Deg, 0) * rotation; Vector3 up = rotation * Vector3.up; #if DRAW_DEBUG_LINES Debug.DrawLine(Vector3.zero, up * 3, Color.green); #endif float yawAngle = Mathf.Atan2(up.x, up.y); const float MaxYawAngle = 150f; // if yaw is greater than MaxYawAngle degrees, it means this is // actually probably a roll > 90 degrees. // go back and fix our numbers for yaw, pitch and roll. if (Mathf.Abs(yawAngle) > MaxYawAngle * Mathf.Deg2Rad) { pitchAngle += (pitchAngle < 0 ? Mathf.PI : -Mathf.PI); rollAngle = (Mathf.Sign(rollAngle) * Mathf.PI - rollAngle); yawAngle = (Mathf.Sign(yawAngle) * Mathf.PI - yawAngle); } // convert angles into -1, 1 axis range, apply dead zone pitch = Mathf.Clamp((PitchAngleOffset - pitchAngle * Mathf.Rad2Deg) / MaxAngle, -1, 1); if (Mathf.Abs(pitch) < DeadZone) pitch = 0; roll = Mathf.Clamp(rollAngle * Mathf.Rad2Deg / MaxAngle, -1, 1); if (Mathf.Abs(roll) < DeadZone) roll = 0; yaw = Mathf.Clamp(-yawAngle * Mathf.Rad2Deg / MaxAngle, -1, 1); if (Mathf.Abs(yaw) < DeadZone) yaw = 0; } }

Below is a replacement for Unity’s AeroplaneUserControl4Axis from the StandardAssets package. If you attach it to the AircraftPropeller game object and remove the AeroplaneUserControl4Axis, you should be able to fly the plane using the controller. You’ll need to make sure there is an OVRManager in the scene so that OVRInput gets updated.

using System.Collections; using System.Collections.Generic; using UnityEngine; using UnityStandardAssets.Vehicles.Aeroplane; [RequireComponent(typeof(AeroplaneController))] public class GearVrAirplaneControl4Axis : MonoBehaviour { // reference to the aeroplane that we're controlling private AeroplaneController m_Aeroplane; private void Awake() { // Set up the reference to the aeroplane controller. m_Aeroplane = GetComponent<AeroplaneController>(); } private void FixedUpdate() { float yaw = 0, pitch = 0, roll = 0; float throttle = 0; bool airbrakes = false; var controller = OVRInput.GetActiveController() & (OVRInput.Controller.LTrackedRemote | OVRInput.Controller.RTrackedRemote); #if UNITY_EDITOR controller = OVRInput.GetActiveController() & OVRInput.Controller.RTouch; #endif if (controller != OVRInput.Controller.None) { var rotation = OVRInput.GetLocalControllerRotation(controller); FlightStickSimulation.CalculateYawPitchRoll(rotation, out yaw, out pitch, out roll); OVRInput.Axis2D primaryAxis = OVRInput.Axis2D.PrimaryTouchpad; #if UNITY_EDITOR primaryAxis = OVRInput.Axis2D.PrimaryThumbstick; #endif throttle = OVRInput.Get(primaryAxis, controller).y; airbrakes = OVRInput.Get(OVRInput.Button.PrimaryIndexTrigger, controller); } // Pass the input to the aeroplane m_Aeroplane.Move(roll, pitch, yaw, throttle, airbrakes); } }

We hope this information is helpful and look forward to exploring other use cases in the future.