In this tutorial, you learn how to build a custom hand pose from scratch by constructing a thumbs up pose for both hands. Since custom hand poses rely on information about your physical hand, like finger position and wrist orientation, custom hand poses aren’t supported for controllers and controller driven hands. For a detailed explanation of how pose recognition works, see Hand Pose Detection.
To try pose recognition in a pre-built scene, see the PoseExamples or DebugGesture scenes.
Note
Hand poses can also rely on velocity and rotation data, which aren't used in this tutorial but are described in Hand Pose Detection.
An outline of the parts that define a thumbs up pose. The pose is a combination of two different shapes (thumb curl open and all fingers closed) and a transform orientation (wrist up).
Hand tracking provides transform data about your hands, like which direction your wrist is facing. To access that data, each hand needs a TransformFeatureStateProvider component.
Under Inspector, add a TransformFeatureStateProvider component.
Repeat these steps for RightHand > HandFeaturesRight.
Set finger thresholds
Finger thresholds define when a finger is in a specific state (ex. curled or open). For the thumbs-up pose, you’ll use the default thresholds included with Interaction SDK. To save time, you’ll copy them from a sample scene.
In a separate Unity window, open the PoseExamples sample scene found under Assets/Samples/Meta XR Interaction SDK/”YOUR SDK VERSION”/Example Scenes/PoseExamples so you can copy the finger thresholds from it.
To import the samples, open the Package Manager window, select Meta XR Interaction SDK, then use the Samples tab and click Import to import sample scenes.
If prompted to import TMP Essentials, import them.
Under Inspector, select the three vertical dots at the top of the FingerFeatureStateProvider component.
A list appears.
In the list, select Copy Component.
Return to the Unity scene you’re using for this tutorial.
In your Unity scene, under Hierarchy, select LeftHand > HandFeaturesLeft.
Under Inspector, select the three vertical dots at the top of the Joints Radius Feature component.
A list of options appears.
In the list, select Paste Component as New.
The FingerFeatureStateProvider component you copied is added to HandFeaturesLeft as a new component.
In the FingerFeatureStateProvider component, set the Hand property to the hand you want to track, LeftHand.
Repeat these steps for RightHand > HandFeaturesRight.
Create an empty GameObject
A pose is contained in an empty GameObject that has a list of unique components, so you need to make an empty GameObject.
Under Hierarchy, add an empty GameObject by right-clicking and selecting Create Empty.
Name the GameObject ThumbsUpPoseLeft.
Under Inspector, add the following components:
ShapeRecognizerActiveState
TransformRecognizerActiveState
ActiveStateGroup
ActiveStateSelector
SelectorUnityEventWrapper
Repeat these steps for the right hand, naming that GameObject ThumbsUpPoseRight.
Define the shapes of the pose
All of the shapes required for a pose are stored in the hand’s ShapeRecognizerActiveState component. The thumbs up pose includes two shapes. The first shape checks that all the fingers are fully closed, and the second shape checks that the thumb is fully extended. Both must be true for the SDK to consider the thumbs up shape as true.
However, a shape doesn’t define a pose on its own because shapes don’t track velocity or the hand’s orientation in space.
Under Hierarchy, select the ThumbsUpPoseLeft GameObject you created in the previous section.
Under Inspector, in the ShapeRecognizerActiveState component, set the Hand property to LeftHand to track the shape of the left hand.
Set Finger Feature State Provider to the HandFeaturesLeft GameObject. If prompted to select a component, select the FingerFeatureStateProvider component. This allows the ShapeRecognizerActiveState component to access the state of all five fingers on the left hand so it can compare them to the shapes you define.
In the Shapes property, click the + twice to add two elements to the list.
Set Element 0 to the FingersAllClosedShapeRecognizer asset by clicking the circle in the property’s textbox and searching for FingersAllClosed. This is the first shape of the pose.
Set Element 1 to the ThumbUpShapeRecognizer asset. This is the second shape of the pose.
Repeat these steps for the ThumbsUpPoseRight GameObject.
Define the pose orientation
Since pose shapes don’t track the orientation of the hand, currently your pose is active whenever your thumb is extended and the other fingers are closed, regardless of which way your hand is rotated. To define the correct orientation, a pose uses the TransformRecognizerActiveState component.
To ensure your pose is a thumbs up and not something similar but incorrect, like a thumbs down, define the pose’s orientation to be the wrist facing upwards in world space.
Under Hierarchy, select the ThumbsUpPoseLeft GameObject.
Under Inspector, in the TransformRecognizerActiveState component, set the Hand property to LeftHand.
Set the Transform Feature State Provider property to HandFeaturesLeft to get the left hand’s transform data.
In the Transform Feature Configs dropdown, select Wrist Up to define the correct orientation as the wrist facing up.
Under Transform Config, in the Up Vector Type dropdown, select World to define the wrist orientation as relative to world space.
Set the Feature Thresholds property to DefaultTransformFeatureStateThresholds by clicking the circle in the property’s textbox and searching for DefaultTransformFeatureStateThresholds. This is a set of default threshold definitions created by the SDK team.
Repeat these steps for the ThumbsUpPoseRight GameObject.
Combine the shapes and orientation
So far you’ve defined the shapes and the orientation that together make a thumbs up. Now it’s time to check the shapes and orientation simultaneously. To check both simultaneously, you’ll combine the shapes and orientation in an Active State Group. This group becomes active when the tracked hand matches all of the required shapes and orientation.
Under Hierarchy, select the ThumbsUpPoseLeft GameObject.
In the Active State Group component, in the Shapes property, click the + twice to add two elements to the list.
In the Active State Group component, add two elements to the list.
Set Element 0 to the ThumbsUpPoseLeft GameObject.
A list of options appears.
Select ShapeRecognizerActiveState from the list.
Set Element 1 to the ThumbsUpPoseLeft GameObject.
A list of options appears.
Select TransformRecognizerActiveState from the list.
Repeat these steps for the ThumbsUpPoseRight GameObject.
Track the state of the pose
To track if the pose is happening, you use the ActiveStateSelector component, which tracks an active state and fires its WhenSelected and WhenUnselected events based on the tracked state.
Under Hierarchy, select the ThumbsUpPoseLeft GameObject.
In the Active State Selector component you added earlier, set the Active State property to the ThumbsUpPoseLeft GameObject.
A list of options appears.
Select ActiveStateGroup from the list since you want to track the combined state of the shapes and orientation, not the individual states.
Repeat these steps for the ThumbsUpPoseRight GameObject.
Choose events to fire
The ActiveStateSelector’s events aren’t exposed in the Unity inspector, so to access these events in the Inspector, you pass the ActiveStateSelector to a SelectorUnityEventWrapper component.
SelectorUnityEventWrapper lets you run custom logic based on the event that’s occurring.
Under Hierarchy, select the ThumbsUpPoseLeft GameObject.
In the Selector Unity Event Wrapper component, set the Selector property to the ThumbsUpPoseLeft GameObject.
Add your own custom functions to the When Selected() list, the When Unselected() list, or both. These functions will trigger when a thumbs up starts or stops.
Repeat these steps for the ThumbsUpPoseRight GameObject.
In this example, the finished thumbs up pose uses an Active State Debug Tree UI component to display the current state of the left hand. If you want to learn how to use the debug tree UI, see the DebugPose scene.
Related topics
To understand the concepts and components used to detect a pose, see Hand Pose Detection.
