Basic Lip Sync Tips

Assessing Lip Sync
Basic Lip Sync Guidelines
Terminology
How To Use Phonemes/Visemes to Perfect a Vub

Assessing Lip Sync

When assessing lip sync, focus on the following questions:

Does the vub integrate well into the overall narrative?
Does the output resemble natural speech?
Do you believe the character is genuinely delivering the dialogue?
Is the output synchronized?
Are any mouth movements absent?
Are any mouth movements extraneous?
Do the lips close and open appropriately with the sounds?

Basic Lip Sync Guidelines

Use the Source media as much as possible by using interpolation.
When a character’s mouth movement does not need to be changed from the original performance, bring the Interpolation down to 0.
When changing interpolation or blendshape values from 100 to 0 or vice versa, ensure the change appears as seamless as possible. We recommend the transition spans at least 4 frames.
Check for things that can’t be happening - remove any physical impossibilities.

Terminology

Key Term	Definition	Examples
phoneme	the smallest unit of sound perceived as meaningful within a given language	In English, /s/ (as in sip) and /z/ (as in zip) are both phonemes, because English speakers derive meaning from them. The presence of /s/ or /z/ can change the meaning of an English word–e.g. If you swap the /s/ in sip with a /z/, it becomes a new word: zip! Though other languages may have both /s/ and /z/ sounds present, if those sounds are interchangeable and do not alter the meaning of a word, then /s/ and/or /z/ would be present as phones rather than phonemes. For instance, in Swedish, there is no /z/ phoneme; so native Swedish speakers often swap /z/ for /s/ when speaking English, because in their native tongue, there is no meaningful difference between /s/ and /z/.
viseme	what a given phoneme or set of phonemes looks like	The phonemes /f/ and /v/ differ in that /f/ is voiceless and /v/ is voiced. Because they are produced so similarly (everything is the same apart from being voiced or voiceless) /f/ and /v/ appear the same visually. Due to their indistinguishable appearance, /f/ and /v/ are often grouped into one viseme category.
articulator	any of the organs or structures involved in speech production—e.g. lips, teeth, tongue, throat, etc.	The articulators involved in /f/ and /v/ phonemes are the: upper lip and lower teeth–which is why these two phonemes are classified as “labiodentals” by linguists!
fricative	a phoneme that requires turbulence or restricted airflow; restriction can be caused by various articulators working together to create a narrow passage to force through	/s/, /z/, /f/, and /v/ are just some examples of fricatives. /s/ and /z/ create restricted airflow by keeping the teeth close together. /f/ and /v/ on the other hand, utilize the upper teeth and lower lip to create restricted airflow.
air	a phoneme that requires turbulence or restricted airflow; restriction can be caused by various articulators working together to create a narrow passage to force AIR through

How To Use Phonemes/Visemes to Perfect a Vub

Let’s break down some of the most common and obvious factors that affect the success of a vub.

Rogue Openings

A rogue opening occurs when the lips part during a moment they should be fully closed. Rogue openings can also be described as false positive lip separations, and they can easily disrupt the believability of a performance. During these sounds, even a small gap between the lips can throw off the realism of your lip sync; so be sure to prioritize removing rogue openings.

To avoid rogue openings, pay careful attention to /m/, /b/, or /p/ sounds:

/m/ requires full lip closure for the entire duration of the sound.
/b/ and /p/ require full lip closure at their onsets, before a burst of air is released.

Gap Errors

A gap error occurs when there is a false positive gap between articulators (lips, teeth, tongue, etc.) that require contact with each other. For example, in order to produce a legible /f/ or /v/ sound, the lower lip must make contact with the upper teeth. When this contact is missed, the /f/ or /v/ loses its readability and harms the performance.

NOTE: Rogue openings technically qualify as a sub-class of gap errors; however, because lip-to-lip contact is so crucial to lip sync success, we gave rogue openings their own category.

Overdrops

An overdrop occurs when there is more jaw opening than what is considered feasible for a given sound. Many consonants, especially fricatives like /s/, /z/, /ʃ/, /ʒ/, etc. require the jaw to be nearly closed. When the jaw is too open, it prevents the articulators from achieving frication (turbulent and noisy airflow produced when air is forced through a narrow passage).

Underdrops

An underdrop occurs when there is less jaw opening than what is considered feasible for a given sound. Vowels typically require more jaw openness compared to consonants. When the jaw is too closed for a vowel it can harm the readability of that vowel; it can also harm the readability of the surrounding consonants by creating improper contrast in the greater context.

Term	What it means	When it matters
rogue opening	when there is a false positive opening between the lips	during p’s, b’s, and m’s
gap error	when there is missed physical contact between key articulators	during f’s and v’s
overdrop	when the jaw is more open than is likely	during most consonants
underdrop	when the jaw is more closed than likely	during vowels