Training Data

What is Training Data

Training Data is additional footage of the character you want to vub. You can upload this into DeepEditor to help generate a more accurate vub.

How Training Data Works

Training data enables DeepEditor to learn how the face of the person you are vubbing moves, so it can produce better results.

DeepEditor will go through and add extra detail to the model based on the video frames in your Training Data.

Training Data Requirements

To get the best results use footage that is really similar to your Source media (e.g. alternate takes) and meets the following criteria:

• The person is speaking and their mouth interior is visible (this helps DeepEditor learn the way the person's mouth moves when they speak)
• The person's face position and angle is very similar to the Source media
• Lighting and background are very similar to the Source media
• There are at least 15 seconds of footage showing a variety of mouth shapes (footage can be in separate video clips)
• There are no black frames.
• Each clip features the same actor as the Source media
• The person's face is clear and unobstructed
• The media is compliant with the DeepEditor Supported Media Formats

Uploading Training Data

You have two options for uploading Training Data:

• Upload to the Media Library first

If you upload your Training Data file to the Media Library first, you can then navigate to your shot, click Add Training Data and select the file from the Media Library.

• Upload from the Shot page

If you choose this option, go to the Shot page, click on Add Training Data, then select one of the following:

• From your computer if uploading files less than 1GB
  • Here, you can drag and drop files or click to find the file and upload it.

• Via Media Shuttle if uploading EXR sequences or files exceeding 1GB.
  • Make sure you have the free Signiant app installed if using this option
  • Drag and drop the folder containing the EXRs and not the individual EXRs themselves. 
  • You can only upload one folder of EXRs at a time.

When you start generating a vub, you will be prompted to select which files you want to use to create your Vub. Based on the vub quality, the character you've selected to vub, and your project timeline, you may choose not to include every training data file you've uploaded.

Here is an example of how adding Training Data can improve your Vub.

Source Media

Vub without Training Data

Vub with Training Data

In the final image, with Training Data, you can see the teeth and the inside of the mouth are much more clearly defined.