Refinement Process Step 5: Mouth Shape Pass
Refine mouth shapes where needed.
It’s important to realize that, in the real world, not every sound is visibly articulated. It’s easy to try to make every sound clearly defined, but this is not the way people talk.
Concentrate on the key sounds and mouth shapes with a focus on making sure they obey the ground rules for being able to form those sounds.
It is recommended to first check and correct the top priority mouth shapes across the whole shot to make sure they are formed correctly. Once checked, render and review your shot.
TOP PRIORITY Mouth Shapes:
After reviewing the top-priority shapes, check any remaining problem areas for these secondary mouth shapes to identify and refine outstanding issues.
SECONDARY Mouth Shapes:
Before starting, make sure you are familiar with the Refinement Best Practices.
Lip Sync Controls
Perfecting mouth shapes in DeepEditor is done using the Lip Sync Controls. These are:
Visemes - intuitive, speech-based mouth shapes commonly used in spoken dialogue.
Advanced (blendshape) controls - individual blendshape controls that can be used to further fine-tune mouth shapes if desired.
How to Use the Lip Sync Controls
Viseme-based refinement is the primary way to adjust mouth shapes and speech performance inside DeepEditor. This approach allows you to work faster and focus on performance rather than technical facial anatomy.
The Advanced Controls should only be used in cases where viseme refinement does not achieve the desired result.
Use visemes when:
- Adjusting dialogue clarity or sync
- Polishing speech performance
- Making quick, readable changes
Use advanced controls when:
- The viseme controls have not achieved the desired result
- Making very specific facial adjustments
- Correcting edge cases
- Layering subtle refinements on top of a viseme
For most workflows, visemes will cover the majority of refinement needs.
In order to use the Visemes and Advanced Controls correctly, you must understand what the correct timing and mouth shapes are for each sound. Below we have provided a detailed guide and videos for each mouth shape. If you cannot achieve the correct timing and shape with Visemes, only then proceed to using the Advanced Controls.
Before starting, make sure you know how to operate the Viseme controls and Advanced Controls.
Assessing Mouth Shapes for Viseme-based Adjustments
Refer to the guide below for the correct timing and mouth shapes for each viseme.
Details of how to use the viseme controls are available here.
M/B/P (e.g. “mat”, “tube”, “cap”)
TIMING AND SHAPES
B & P:- The lips should be pressed together on the frame before the sound (and possibly the frame before that too, depending on the strength of the B or P).
- The lips must release from each other when the B & P sounds start.
M:
- The M sound occurs only while the lips are closed.
- The lips release when the next sound occurs.
Need to use the Advanced controls? See the guide and tutorial here.
F/V (e.g. “fate”, “turf”, “velcro”)
TIMING AND SHAPES
- F & V sounds only occur while the bottom lip is in contact with the top teeth.
- No lower teeth should be visible.
Need to use the Advanced controls? See the guide and tutorial here.
W/Oo (e.g. “however”, “when”, “cool”, “move”)
TIMING AND SHAPES
- Oo and W sounds occur with a rounding of the lips and a small lip opening.
- The Oo sound is heard while the mouth shape is held.
- The W mouth shape is strongest on the frame before the peak of the sound as it releases into the next sound.
Need to use the Advanced controls? See the guide and tutorial here.
S/Z/T/D (e.g. “sausages”, “hazy”, “potatoes”, “fades”)
TIMING AND SHAPES
- Using Audio Scrubbing, find the frame or frames where you hear the ‘hiss’ of the S or Z sounds, or the frame before you hear the ‘pop’ of the T or D.
- Typically, the mouth shape for S/Z is made while the sound is made.
- Typically the mouth shape for T/D is made the frame before the sound is made.
- The lips must not be fully sealed
- There should ideally be some visibility of the teeth with no visible gap between them.
- The jaw for T/D sounds can sometimes be fractionally more open than for S/Z sounds.
Need to use the Advanced controls? See the guide and tutorial here.
Th (e.g. “thick”, “path”)
TIMING AND SHAPES
- The Th sound only occurs while forming the Th mouth shape.
- The Th shape involves the tongue touching the underside of the top teeth, meaning that it can peek out between the teeth. DeepEditor does not currently control the tongue, so to make a Th shape work, ensure the jaw is open just enough to allow for a hypothetical tongue peeking out.
Need to use the Advanced controls? See the guide and tutorial here.
Ch/Sh/Dge/J (e.g. “channel”, “match”, “clash”, “fudge”, “ajar”)
TIMING AND SHAPES
- The lips should not be sealed.
- There should be little to no gap between the teeth.
- The lips often funnel outward.
- These sounds are typically heard as the mouth is starting to release from the position, except for elongated Sh sounds.
Need to use the Advanced controls? See the guide and tutorial here.
R (e.g. “crash”, “erode”)
TIMING AND SHAPES
- The lips should curl slightly outwards.
- There should be a slight rounding of the lips.
- The opening between the lips should be small
- The peak of a R mouth shape should be the frame before we hear the peak of the sound – which occurs as the shape is released.
Need to use the Advanced controls? See the guide and tutorial here.
N/L (e.g. “panic”, “new”, “milk”, “ballast”)
TIMING AND SHAPES
- The lips should not be sealed.
- There should be a small visible gap between the teeth. N/L is formed by the tongue touching the roof of the mouth, so we focus the jaw opening to make this physically possible.
- The N and L sounds are heard when the mouth releases from this position.
Need to use the Advanced controls? See the guide and tutorial here.
Ah (e.g. “mat”, “flap”)
TIMING AND SHAPES
- The peak of the Ah sound should match the peak of the jaw opening.
- The lips and jaw must not be closed.
- The size of the jaw opening should match the performance size.
Need to use the Advanced controls? See the guide and tutorial here.
Ee (e.g. “heed”, “happily”)
TIMING AND SHAPES
- There should be a light jaw opening, leaving a slight separation between the teeth
- There should be an increased mouth width; it can look like a smile.
- The peak of the mouth shape should match the peak of the sound.
Need to use the Advanced controls? See the guide and tutorial here.
Download the complete Mouth Shapes Pass Guide:
Mouth Shapes Pass Guide [PDF / Light Design]
Mouth Shapes Pass Guide [PDF / Dark Design]
More downloadable guides on Refinement are available here.
Read Next: Refinement Troubleshooting & FAQs