Apr 12, 2017

Tech Notes: Aircraft Controls Using the Gear VR Controller

Trevor Dasch Blog Hero Image

With the upcoming release of the new Gear VR Controller, people will be able to use their hand to point, drag and drop, tilt, shoot, and more in Gear VR apps. We’re discovering some very interesting use cases for the new Gear VR Controller, like controlling aircraft and spacecraft. The usual input for an aircraft is in the form of yaw, pitch, and roll. Yaw is the left-right rotation of the aircraft, pitch is up-down tilt, and roll is the length-wise rotation where one wing goes up and the other goes down. Generally, real aircraft have more control over pitch and roll, which are controlled by the wing flaps, than yaw, which is controlled by the tail rudder.

Real aircraft are either controlled using a yoke (steering wheel thing) or a stick. Fans of flight simulators are very familiar with flight sticks or HOTAS (hands on throttle and stick), where they’re able tilt the stick to input pitch and roll—and sometimes twist the stick to input yaw. These mechanical sticks are able to read the axes from the physical device and send it to the software, which then apply the yaw, pitch, and roll adjustments to the aircraft.

Because the Gear VR Controller provides an orientation instead of a three-axis input, in order to use it as a flight stick we’ll have to do some math to convert a rotation into yaw, pitch, and roll. You might be tempted to just grab the Euler angles from the rotation and call it a day, but this will cause some issues. Because a rotation can be expressed as different sets of Euler angles, you may find the controls to be unpredictable, and if you tilt left, there may be a point where controls suddenly swing to the right or vice-versa. Generally you want to avoid this, so we’re going to use vector math to calculate the specific angles we want.

To start breaking down the problem, it makes sense to think of the current controller rotation as the result of the sum of three hinges. The first hinge goes forward and backward and controls the pitch. This value can be found by projecting the current controller forward onto the YZ plane, then determining the rotation of this vector around the X axis. Fortunately, Unity provides a function to get this projection: Vector3.ProjectOnPlane(). Once we have the projection, we can find the angle using the arctangent of y/z of this new vector. There is one gotcha however. If the controller forward is parallel to the X axis, the projection will be zero, which means you won’t be able to get an angle. To avoid this, if I detect that the projection is zero, I use either the controller right vector or left vector, depending on if the controller is turned left or right (whichever is facing “forward” if the controller is sitting flat on a surface). The one downside of using this method is it will cause gimbal lock with yaw. Arctangent will return a value between -90 and 90 degrees, while we want a value in the 360 range. Fortunately, Unity has a function Mathf.Atan2() which lets you pass in your y and z separately, and will handle divide by zero and determining a result in the full 360 degree range.

After calculating pitch, you can multiply your original rotation by the inverse of this value, which will place your forward vector into the XZ plane. You can then grab the roll rotation using Atan2(forward.x, forward.z). If you then multiply your rotation by the inverse of the roll rotation, your forward vector will now be Vector3(0,0,1) or the world forward direction.

Now let’s imagine our yaw component as determined by a twist of the stick. We’ve moved the stick to be in its upright position, so the only rotation remaining is this twist rotation. Determining it is as simple as getting our up vector from the rotation and grabbing the arctangent of x/y. In the case where the controller is pointed exactly 90 degrees right, the yaw will always be zero. Also, without the physical resistance of a flight stick, it’s very hard to tell at certain angles whether you’re twisting the stick at all. For that reason, it might make sense to drop the yaw component altogether, since you can fly a plane using only pitch and roll.

One downside to this method is that it only lets you get a roll angle of up to 90 degrees. Once you roll past that, it reads as a 180-degree pitch forward with a 180-degree yaw. To allow greater roll, you can detect any yaw over 90 degrees (or some higher threshold if you want to allow larger yaws) and assume that was a mistake. You can then reverse the 180-degree rotation and determine the larger roll value.

Below is an implementation of these controls—they take a quaternion and produce a yaw, pitch, and roll value between -1 and 1.

    #define DRAW_DEBUG_LINES
    
    using System.Collections;
    using System.Collections.Generic;
    using UnityEngine;
    
    public static class FlightStickSimulation
    {
        private const float MaxAngle = 60f;
        private const float DeadZone = 0.2f;
        private const float PitchAngleOffset = 45f;
    
        public static void CalculateYawPitchRoll(Quaternion rotation, out float yaw, out float pitch, out float roll)
        {
            Vector3 forward = rotation * Vector3.forward;
    
            var pitchProj = Vector3.ProjectOnPlane(forward, Vector3.right);
    
            if (pitchProj.sqrMagnitude < 0.001f)
            {
                forward = rotation * (forward.x > 0 ? Vector3.left : Vector3.right);
                pitchProj = Vector3.ProjectOnPlane(forward, Vector3.right);
            }
    
    #if DRAW_DEBUG_LINES
            Debug.DrawLine(Vector3.zero, forward * 3, Color.blue);
    #endif
  
            float pitchAngle = Mathf.Atan2(pitchProj.y, pitchProj.z);
    
            // cancel out pitch
            rotation = Quaternion.Euler(pitchAngle * Mathf.Rad2Deg, 0, 0) * rotation;
  
            forward = rotation * Vector3.forward;
  
    #if DRAW_DEBUG_LINES
            Debug.DrawLine(Vector3.zero, forward * 3, Color.red);
    #endif
  
            var rollProj = Vector3.ProjectOnPlane(forward, Vector3.up);
  
            float rollAngle = Mathf.Atan2(rollProj.x, rollProj.z);
    
            // cancel out roll
            rotation = Quaternion.Euler(0, -rollAngle * Mathf.Rad2Deg, 0) * rotation;
    
            Vector3 up = rotation * Vector3.up;
    
    #if DRAW_DEBUG_LINES
            Debug.DrawLine(Vector3.zero, up * 3, Color.green);
    #endif
    
            float yawAngle = Mathf.Atan2(up.x, up.y);
    
            const float MaxYawAngle = 150f;
            // if yaw is greater than MaxYawAngle degrees, it means this is 
            // actually probably a roll > 90 degrees.
            // go back and fix our numbers for yaw, pitch and roll.
            if (Mathf.Abs(yawAngle) > MaxYawAngle * Mathf.Deg2Rad)
            {
                pitchAngle += (pitchAngle < 0 ? Mathf.PI : -Mathf.PI);
                rollAngle = (Mathf.Sign(rollAngle) * Mathf.PI - rollAngle);
                yawAngle = (Mathf.Sign(yawAngle) * Mathf.PI - yawAngle);
            }
    
            // convert angles into -1, 1 axis range, apply dead zone
            pitch = Mathf.Clamp((PitchAngleOffset - pitchAngle * Mathf.Rad2Deg) / MaxAngle, -1, 1);
            if (Mathf.Abs(pitch) < DeadZone)
                pitch = 0;
    
            roll = Mathf.Clamp(rollAngle * Mathf.Rad2Deg / MaxAngle, -1, 1);
            if (Mathf.Abs(roll) < DeadZone)
                roll = 0;
    
            yaw = Mathf.Clamp(-yawAngle * Mathf.Rad2Deg / MaxAngle, -1, 1);
            if (Mathf.Abs(yaw) < DeadZone)
                yaw = 0;
        }
    }

Below is a replacement for Unity’s AeroplaneUserControl4Axis from the StandardAssets package. If you attach it to the AircraftPropeller game object and remove the AeroplaneUserControl4Axis, you should be able to fly the plane using the controller. You’ll need to make sure there is an OVRManager in the scene so that OVRInput gets updated.

using System.Collections;
    using System.Collections.Generic;
    using UnityEngine;
    using UnityStandardAssets.Vehicles.Aeroplane;
    
    [RequireComponent(typeof(AeroplaneController))]
    public class GearVrAirplaneControl4Axis : MonoBehaviour {
    
        // reference to the aeroplane that we're controlling
        private AeroplaneController m_Aeroplane;
    
        private void Awake()
        {
            // Set up the reference to the aeroplane controller.
            m_Aeroplane = GetComponent<AeroplaneController>();
        }
    
    
        private void FixedUpdate()
        {
            float yaw = 0, pitch = 0, roll = 0;
            float throttle = 0;
            bool airbrakes = false;
    
            var controller = OVRInput.GetActiveController() & (OVRInput.Controller.LTrackedRemote | OVRInput.Controller.RTrackedRemote);
    
    
            #if UNITY_EDITOR
                controller = OVRInput.GetActiveController() & OVRInput.Controller.RTouch;
            #endif
    
            if (controller != OVRInput.Controller.None)
            {
                var rotation = OVRInput.GetLocalControllerRotation(controller);
    
                FlightStickSimulation.CalculateYawPitchRoll(rotation, out yaw, out pitch, out roll);
    
                OVRInput.Axis2D primaryAxis = OVRInput.Axis2D.PrimaryTouchpad;
    
                #if UNITY_EDITOR
                    primaryAxis = OVRInput.Axis2D.PrimaryThumbstick;
                #endif
    
                throttle = OVRInput.Get(primaryAxis, controller).y;
                airbrakes = OVRInput.Get(OVRInput.Button.PrimaryIndexTrigger, controller);
            }
    
            // Pass the input to the aeroplane
            m_Aeroplane.Move(roll, pitch, yaw, throttle, airbrakes);
    
        }
    }

We hope this information is helpful and look forward to exploring other use cases in the future.

Unity

Explore more

Mar 27, 2025Quarterly Developer Recap: Tracked Keyboard, Hand Gestures, WebXR PWAs and more

Unlock the latest updates for developers building 3D, 2D and web-based experiences for Horizon OS, including the new tracked keyboard, microgesture-driven locomotion, and more.

Apps, Design, Locomotion, Native SDK, Quest, Unity, WebXR

Read article

Mar 19, 2025GDC 2025: Strategies and Considerations for Designing Incredible MR/VR Experiences

Dive into design and rendering tips from GDC 2025. Hear from the teams behind hit games like Batman: Arkham Shadow and Demeo and learn how you can make your mixed reality and VR games more enjoyable and accessible.

Accessiblity, All, Apps, Avatars, Design, GDC, Games, Hand Tracking, Optimization, Quest, Unity

Read article

Mar 18, 2025GDC 2025: Emerging Opportunities in MR/VR on Meta Horizon OS

Discover some of the opportunities we shared at GDC 2025 to accelerate your development process, level up your skillset and expand your reach with Meta Horizon OS.

Apps, GDC, Games, Marketing, Multi-User, Oculus Start, Quest, Unity, Unreal

Read article