On this Page
Expressive features give Oculus Avatars more advanced facial geometry, allowing for realistic and nuanced animation of various facial behaviors. Expressive features increase social presence and make interactions seem more natural and dynamic. Expressive features are comprised of the following:
For the code to add an avatar with expressive features to an Unreal project, see the “Launching the Avatar Samples Unreal Project” section of Unreal Getting Started Guide.
For expressive lip-syncing, you will need to download and integrate the OvrLipSync plugin into your project’s plugin folder and update your uproject to use it. See AvatarSamples, where it’s currently included, or download the latest from the Oculus developer downloads page.
To see expressive features in a sample, build and run AvatarSamples. In the Unreal Editor, you will see several new options exposed to configure the avatar. Make sure that Enable Expressive is set.
Keep the following considerations in mind concerning materials:
As an avatar’s gaze shifts, different gaze targets enter and leave its field of view, and its eyes automatically move between and focus on valid gaze targets. See the “Gaze Modeling” section below for more information on gaze targeting.
Any object in a scene can be made a gaze target by doing the following:
An object of interest can be tagged as a gaze target by attaching a UOvrAvatarGazeTarget to an object. This is exposed through the editor for convenience. There are four types of gaze targets, listed here in descending order of saliency:
Unless otherwise specified in by code using
void SetGazeTransform(USceneComponent* sceneComp) the transform will point to the root object scene component that it is attached. It may desirable to tune that on larger objects to refine the point of interest.
The following sections provide more information on each of the expressive features.
OVRLipsync uses voice input from the microphone to drive realistic lip-sync animation for the avatar. Machine-learned viseme prediction is applied to the input and used to translate it into a set of blend shape weights used to animate the mouth in real-time. Physically based blending is used to produce more natural and dynamic mouth movement, along with subtle facial movements around the mouth.
Gaze modeling enables an avatar’s eyes to look around and exhibit gaze dynamics and patterns developed by studying human behavior. Here are the currently implemented kinds of eye behavior:
Gaze targets are specifically tagged objects that represent a visual point of interest for an avatar’s gaze. As an avatar’s gaze shifts, different gaze targets enter and leave its field of view, and its eyes will automatically move between and focus on valid gaze targets using the behaviors listed above. If no gaze targets are present, an avatar will exhibit ambient eye movement behavior.
When presented with multiple gaze targets, the targeting algorithm that determines which gaze target an avatar looks at calculates a saliency score to each target. This score is based on direction and velocity of head movement, eccentricity from the center of vision, distance, target type, and whether the object is moving.
Blink modeling enables an avatar to simulate human blinking behaviors, such as blinking to keep their eyes clean and at the end of spoken sentences. Blinks may also be triggered by sweeping eye movements.
Expression modeling enables slight facial nuances and micro-expressions on an avatar’s face to increase social presence when it isn’t actively speaking. These expressions are minor to avoid the implication they are indicating an actual mood or response to anything happening around them.
You can test expressive features by using the following user IDs: