Priority Rationale
When it comes to lip sync, not all phonemes were created equal - some are more noticeable to the audience.
When refining the lip sync of a shot, start with the highest priority shapes. Then assess the lip sync before moving onto the lower priority mouth shapes.
Below you can read specific guidance on using blendshapes in the DeepEditor refine tool to achieve the right look.
Highest Priority Mouth Shapes
Lower Priority Mouth Shapes
M/B/P Shapes
For natural-looking B & P sounds, ensure the lips are closed at least one frame before the sound. The B & P sounds occur after the lips release.
Observe how you say “beet,” “boat,” “pie,” and “pat.”
Ms take place only during lip closure. The M sound occurs only while the lips are pressed together. The lips release when the next sound occurs.
Observe how you say "me," "mat," and "mystic."
Alter the DeepEditor timeline to ensure there is no visible gap between the top and bottom lips at the correct frame(s) for M, B, and P sounds with the following blendshapes:
Recommended Blendshapes for M/B/P Shapes
Jaw - Open (Negative Value)
-
- Close the jaw with a negative value that brings the bottom lip up to meet the top lip. Use in tandem with ‘Mouth Shrug - Upper’ so that the lips come together equally until lip closure is achieved.
Mouth Shrug - Upper (Negative Value)
-
- Lower the top lip with a negative value that brings it down to meet the bottom lip. Use in tandem with ‘Jaw - Open’ so that the lips are coming together equally until lip closure is achieved.
Optional Blendshape for M/B/P Shapes
Mouth - Funnel (Negative Value)
-
- Pull the lips inwards and slightly together with a gentle negative value. This can help with pressing the lips together.
F/V Shapes
For natural-looking F & V sounds, ensure that the lower lip is covering at least part of the upper teeth. F & V sounds only occur while the bottom lip is in contact with the top teeth. No lower teeth should be visible.
Observe how you say the words “fate,” “fly,” “vice,” and “velcro.”
Close the jaw until the bottom lip has sufficient contact with the top teeth. Lift the top lip—if necessary—away from the bottom lip such that the lips are not sealed. If helpful, curl the bottom lip a little towards the top teeth.
Recommended Blendshapes For F/V Shapes
Jaw - Open (Negative Value)
-
- Close the jaw with a negative value so that the bottom lip covers the lower teeth.
Mouth Shrug - Upper
-
- This blendshape will raise the top lip. Once the jaw has been adjusted such that the bottom lip is in contact with the top teeth, ensure that the lips are not touching by raising the top lip.
Optional Blendshapes For F/V Shapes
Mouth Roll - Lower
-
- To bring the lower lip inward, creating the illusion that the lower lip is pressing against the bottom teeth.
Mouth - Funnel
-
- Mouths can purse together when saying F or V. ‘Mouth - Funnel’ will create this slightly rounded shape.
W/Oo Shapes
This viseme is characterized by lip rounding with a small lip opening. The lip opening varies somewhat in size depending on how tensed the lips are.
Observe how you say the words “too,” “moon,” and “aloof.”
The Oo sound is heard while the mouth shape is held.
Observe how you say the words “waste,” “war,” and “whip.”
The peak of a W sound is heard just as it releases into the next sound.
Recommended Blendshapes for W/Oo Shapes
Jaw - Open (Negative)
-
- If the mouth looks too open, slightly close the jaw. However, the mouth should never be fully closed during W/Oo shapes
Mouth - Pucker
-
- To create a rounded shape, or at least shapes that are more rounded compared to the surrounding shapes.
Mouth Shrug - Upper (Negative)
-
- To slightly lower the upper lip to bring the lips closer together so that there is little to no visibility of the top teeth.
Th Shapes
At a basic level, the Th shape involves the tip of the tongue peeking out between the upper and lower teeth.
Observe how you say the words “that," "smooth," and "breath."
DeepEditor does not currently control the position of the tongue, so to make this feature believable, ensure the jaw is not fully closed or too closed to allow for a hypothetical tongue protrusion.
Recommended Blendshapes for Th Shapes
Jaw - Open
-
- If the Jaw is too closed, in a way that air could not believably escape, increase Jaw - Open
Mouth Shrug - Upper
-
- If the lips still look too closed, consider adding ‘Mouth Shrug - Upper’ to lift the upper lip slightly.
Optional Blendshape for Th Shapes
Mouth - Funnel (Negative)
-
- ‘Th’ shapes may cause the mouth corners to stretch. Add a negative value to ‘Mouth - Funnel’ to increase the lip corner-to-lip corner distance.
S/Z/T/D Shapes
The lips must not be fully closed, and the jaw must not be significantly open. The believability relies on the teeth being showcased. There should be no visible gap between the teeth.
Observe how you say the words "sad," "zebra," “table,” and "den."
Although the tongue moves differently in natural speech, the close proximity of the upper and lower teeth is what creates these shapes.
Recommended Blendshapes for S/Z/T/D Shapes
Jaw - Open (Negative)
-
- Use a negative value of ‘Jaw - Open’ to close the jaw if needed.
Mouth Shrug - Upper
-
- After setting the jaw closure correctly, ensure the lips do not seal. Use ‘Mouth Shrug - Upper’ to part the lips slightly.
Ch/Sh/Dge/Zh/J Shapes
Ch/Sh/Dge/Zh/J sounds require open lips and little to no gap between the teeth. In many cases, you may notice the lips funnel or protrude outward. The general idea here is to funnel the lips and then adjust the jaw so it’s nearly closed but not fully.
Observe how you say the words "change," "shovel," "dodge," and "jump."
These sounds are typically heard as the mouth is starting to release from the position, except for elongated Sh sounds.
Recommended Blendshapes for Ch/Sh/Dge/Zh/J Shapes
Jaw - Open (Negative)
-
- If the jaw is too open for nearly touching teeth, use a negative value of ‘Jaw - Open’ to achieve the correct look.
Mouth - Funnel
-
- If the lips do not protrude outward in a natural way for these sounds, add ‘Mouth - Funnel’
Optional Blendshape for Ch/Sh/Dge/Zh/J Shapes
Mouth Shrug - Upper
-
- These shapes sometimes show the top row of teeth. You could shift the top lip up with ‘Mouth Shrug - Upper’ if you need more teeth showing. Conversely, you can lower the top lip with a negative value if ‘Mouth - Funnel’ has overexposed the top teeth.
N/L Shapes
In general, the N & L mouth shape requires parted lips. There should be a very small visible gap between the teeth.
N & L sounds are formed mainly by the tongue touching the roof of the mouth, which is not visible during dialogue, so this is why we focus on just the visible gap between the lips.
Observe how you say the words "lip," "nest," "lone," and "laugh."
The N and L sounds are heard when the mouth releases from this position.
Recommended Blendshape for N/L Shapes
Jaw - Open
-
- If there is no visible gap between the teeth, open the jaw to create a small gap.
Ah/Eh Shapes
The values may change, but the general idea is that the lips and jaw must not be closed. The jaw should be opened depending on the context.
Observe how you say the words "bath," “arm,” "net," and "then."
Recommended Blendshape:
Jaw - Open
-
- Open the jaw to make sure the opening is apparent compared to the surrounding sounds.
Optional Blendshape for Ah/Eh Shapes
Mouth - Pucker (Negative)
-
- Avoid rounding of the mouth. Avoid a mouth shape that looks similar to Oo.
Ee Shapes
The Ee shape involves a light jaw drop, leaving a slight separation between upper and lower teeth and an increased lip corner-to-lip corner distance. It can look like a smile.
Observe how you say the words "bee," "eel," and "teeth."
Recommended Blendshapes for Ee Shapes
Mouth - Pucker (Negative)
-
- Increase the distance between the corners of the mouth
Mouth Smile - Left & Mouth Smile - Right
-
- The corners of the mouth frequently upturn when saying an ‘ee’ sound.
- ‘Mouth Smile - Left’ and ‘Mouth Smile - Right’ will commonly have the same numeric values.
Optional Blendshapes for Ee Shapes
Jaw - Open
-
- Use if the slight separation between the upper and lower teeth is not apparent.
Mouth - Funnel (Negative)
-
- Using a small negative value of ‘Mouth - Funnel’ can create the look of the lips pressing against the teeth that occurs when saying an “ee” sound.
- A negative value of ‘Mouth - Funnel’ also brings the lips closer together. This can be used when the separation between the upper and lower teeth is too wide.
R Shapes
This viseme is characterized by lips curling slightly outwards and a slight rounding with a small lip opening.
Observe how you say the words "red," “right,” and “razor.”
Be mindful that the shape of the R changes in relation to its position in a word: at the beginning, it is more rounded, whereas at the end of a word or a syllable, it is more relaxed.
Observe how you say the words "father," "armor," and "car."
Recommended Blendshapes for R Shapes
Jaw - Open (Negative)
-
- If the mouth looks too open, slightly close the jaw. However, the mouth should never be fully closed during R shapes
Mouth Shrug - Upper
-
- To slightly raise the upper lip and part the lips.
Mouth - Funnel
-
- Can be used to round the mouth without pursing the lips with ‘Mouth -Pucker.’
Optional Blendshape for R Shapes
Mouth - Pucker
-
- To create a rounded shape, or at least contrastive shapes that are more rounded compared to the surrounding shapes.
