What is a VFX Turnover Package?
The VFX turnover package is one of the export options available for Final quality vubs. It is meant to provide support for taking DeepEditor’s outputs to a traditional VFX package like Nuke, Flame, or AfterEffects to make any necessary modifications.
Prerequisites:
1. Generate a Final vub with your highest resolution media.
2. Complete all refinement work in the Refinement tool before exporting the VFX Turnover package and commencing your VFX clean-up work.
3. Signiant app installed to allow downloads via Media Shuttle.
The VFX package currently contains the following data:
A multi-layer OpenEXR image sequence with the frames that have been generated in DeepEditor. Each frame contains 3 layers.- An RGBA image with the final result generated by DeepEditor, containing only the pixels that DeepEditor created (currently the lower half of the face). This way, these images can be easily composited on top of the original frames.
- The output of DeepEditor’s neural renderer without any compositing applied, allows VFX artists to apply their compositing processes if desired.
- An ST map that can be used to wrap any textures or masks and composite them on the output.
- Using the Stabilisation_Flw Nuke gizmo, an artist can generate a Nuke CornerPin node, stabilising the Render. You can download this node here (download link).
- Interpolation Data. A per-frame value ranging from zero to one. This interpolation value represents the difference in mouth shape between the source and the visually dubbed output. A value of one indicates the visual dub is fully active, while a value of zero suggests the original source can be revealed, disabling the render. Using the Interpolation_PT_Flw Nuke Gizmo, an artist can extract these values into a curve. You can download this node here (download link).
Note: The OpenEXR files are in ACES 2065-1 color space.
Note: Image sequences contain only the frames where a face has been detected and processed in DeepEditor. For example, if a 100-frame shot was provided as an input, but the character only appears in 90 frames, then the VFX package will contain only these 90 frames, labelled correctly with their respective frame numbers in the original sequence.
- 2 non-modifiable 3D meshes, one of the face from the original source, and another of the DeepEditor output. Both meshes are in the Alembic file format. This gives a VFX artist even more control, allowing them to project masks and textures on the mesh that can then be composited on top of the DeepEditor output.
- A 3D camera representing the camera angle and movements of the original shot.
Note: The camera, which is output as part of the alembic package, is only used to view/render the face geometry and is not a full scene tracked camera. In other words, it bears no relation to the actual shot camera.
These are the current contents of the package. Improvements and extra elements are planned in the future.
A more comprehensive set of tooling and documentation can be downloaded here (ReadMe can be found in the .zip)
How to Export the VFX Turnover Package
Option 1: Export VFX Turnover Package from Vubs Page
Option 2: Export VFX Turnover Package from Refine Page
Once the package is exported, you will find it in the Exports page.
Like any other export, you will need to get consent before you can download it (unless your project type is Localisation or your Rights Clearance Manager has configured your Editing Project settings to NOT require consent).
Once your consent request is approved, you should be able to select the Download No Watermark option which will then show a dialog that takes you to MediaShuttle to complete the download.
Note: When you try to download a VFX package for the first time, you might get a message saying that the link is still being generated and you should come back again later. Usually, the link is ready within a couple of minutes, so if you click the Download No Watermark button again, it should work.
Guidance on the use of the VFX Turnover Package in software like Nuke can be found in this video.