Digital Compositing for Film and Video, 4th Edition

Digital Compositing for Film and Video, 4th Edition

by Steve Wright
Released November 2017
Publisher(s): Routledge
ISBN: 9781315283999

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Written by senior compositor, technical director and master trainer Steve Wright, this book condenses years of production experience into an easy-to-read and highly-informative guide suitable for both working and aspiring visual effects artists.

This expanded and updated edition of Digital Compositing for Film and Video addresses the problems and difficult choices that professional compositors face on a daily basis with an elegant blend of theory, practical production techniques and workflows. It is written to be software-agnostic, so it is applicable to any brand of software. This edition features many step-by-step workflows, powerful new keying techniques and updates on the latest tech in the visual effects industry.

Workflow examples for:

  • Grain Management

  • Lens Distortion Management

  • Merging CGI Render Passes

  • Blending Multiple Keys

  • Photorealistic Color Correction

  • Rotoscoping

Production Techniques for:

  • Keying Difficult Greenscreens

  • Replicating Optical Lens Effects

  • Advanced Spill Suppression

  • Fixing Discoloured Edges

  • Adding Interactive Lighting

  • Managing Motion Blur

With brand new information on:

  • Working in linear

  • ACES Color Management

  • Light Field Cinematography

  • Planar Tracking

  • Creating Color Difference Keys

  • Premultiply vs. Unpremultiply

  • Deep Compositing

  • VR Stitching

  • 3D Compositing from 2D Images

  • How Color Correction ops Effect Images

  • Color Spaces

  • Retiming Clips

  • Working with Digital Cinema Images

  • OpenColorIO

A companion website offers images from the examples discussed in the book allowing readers to experiment with the material first-hand.

Table of contents

  1. Cover
  2. Title
  3. Copyright
  4. Table of Contents
  5. About the Author
  6. Acknowledgements
  7. Preface
  8. Chapter 1 – Getting Started
    1. 1.1 How this Book is Organized
    2. 1.2 Web Content
    3. 1.3 What’s New in the 4th Edition
    4. 1.4 Gold Mines
    5. 1.5 Tool Conventions
      1. 1.5.1 The Slice Tool
      2. 1.5.2 Flowgraphs
      3. 1.5.3 Color Lookup Tables (LUTs)
      4. 1.5.4 Nuke
    6. 1.6 Data Conventions
      1. 1.6.1 Floating Point Data
        1. 1.6.1.1 Banding
        2. 1.6.1.2 Clipping
      2. 1.6.2 Linear Light Space
      3. 1.6.3 HDR Images
      4. 1.6.4 Stops
  9. PART I MAKING A GREAT COMPOSITE
    1. Chapter 2 – Pulling Keys
      1. 2.1 Luma Keys
        1. 2.1.1 How Luma Keys Work
        2. 2.1.2 Making Your Own Luminance Image
          1. 2.1.2.1 Variations on the Luminance Equations
          2. 2.1.2.2 Non-luminance Monochrome Images
        3. 2.1.3 Making Your Own Luma Keyer
      2. 2.2 Chroma Keys
        1. 2.2.1 How Chroma Keys Work
        2. 2.2.2 Making Your Own Chroma Keyer
        3. 2.2.3 Making a 3D Chroma Keyer
      3. 2.3 Difference Mattes
        1. 2.3.1 How Difference Mattes Work
        2. 2.3.2 Making Your Own Difference Matte
          1. 2.3.2.1 Making the Difference Image
          2. 2.3.2.2 Making the Difference Matte
      4. 2.4 Bump Mattes
      5. 2.5 Color Difference Keys
      6. 2.6 The “Blur and Grow” Technique
      7. 2.7 Rotoscoping
        1. 2.7.1 Control Point Coherency
        2. 2.7.2 Shape Breakdown
          1. 2.7.2.1 Hierarchical Articulation
          2. 2.7.2.2 Organization
        3. 2.7.3 Bezier or B-spline?
        4. 2.7.4 Keyframe Strategies
          1. 2.7.4.1 On 2s
          2. 2.7.4.2 Binary Multiples
          3. 2.7.4.3 Bifurcation
          4. 2.7.4.4 Motion Extremes
        5. 2.7.5 Motion Blur
          1. 2.7.5.1 Spline Placement
          2. 2.7.5.2 Edge Decontamination
        6. 2.7.6 Inspection
    2. Chapter 3 – Working with Keyers
      1. 3.1 Keyers
      2. 3.2 How Keyers Work
        1. 3.2.1 Calculating the Color Difference Matte
          1. 3.2.1.1 The Theory
          2. 3.2.1.2 Pulling the Raw Matte
          3. 3.2.1.3 A Simplified Example
          4. 3.2.1.4 A Slightly More Realistic Case
          5. 3.2.1.5 And Now, the Real World
          6. 3.2.1.6 Matte Edge Penetration
        2. 3.2.2 Scaling the Raw Matte
      3. 3.3 The After Effects Keyer
        1. 3.3.1 Step-by-step Procedure
        2. 3.3.2 Flowgraph of the After Effects Keyer
      4. 3.4 Typical Greenscreen Problems
        1. 3.4.1 Overexposed
        2. 3.4.2 Underexposed
        3. 3.4.3 Impure Greenscreens
        4. 3.4.4 Uneven Lighting
      5. 3.5 Preprocessing the Greenscreen
        1. 3.5.1 Denoise and Degrain
        2. 3.5.2 Screen Leveling
        3. 3.5.3 Local Suppression
        4. 3.5.4 Channel Clamping
        5. 3.5.5 Channel Shifting
        6. 3.5.6 Screen Correction
          1. 3.5.6.1 Step-by-step Procedure
          2. 3.5.6.2 Pictographic Flow Chart
          3. 3.5.6.3 Flowgraph of the Screen Correction Procedure
          4. 3.5.6.4 How to Create a Clean Greenscreen
    3. Chapter 4 – Refining Mattes
      1. 4.1 Gamma Slamming
      2. 4.2 Garbage Mattes
        1. 4.2.1 Pre-matting
        2. 4.2.2 Post-matting
      3. 4.3 Filtering the Matte
        1. 4.3.1 Noise Suppression with a Median Filter
        2. 4.3.2 Softer Edges
        3. 4.3.3 Controlling the Blur Operation
          1. 4.3.3.1 The Blur Radius
          2. 4.3.3.2 The Blur Percentage
          3. 4.3.3.3 Masking the Blur
      4. 4.4 Adjusting the Matte Size
        1. 4.4.1 Eroding a Matte with Blur and Scale
        2. 4.4.2 Dilating a Matte with Blur and Scale
        3. 4.4.3 Blurring Out
        4. 4.4.4 Sculpting Edges
      5. 4.5 Edge Masks
    4. Chapter 5 – Spill Suppression
      1. 5.1 Sources of Spill
      2. 5.2 The Despill Operation
      3. 5.3 Despill Algorithms
        1. 5.3.1 Green Limited by Red
          1. 5.3.1.1 Implementing the Algorithm
          2. 5.3.1.2 The Spillmap
        2. 5.3.2 Green Limited by the Average of Red and Blue
        3. 5.3.3 An Adjustable Despill
        4. 5.3.4 What About Blue Spill?
        5. 5.3.5 Refining the Despill
          1. 5.3.5.1 Channel Shifting
          2. 5.3.5.2 Spillmap Scaling
          3. 5.3.5.3 Mixing Despills
          4. 5.3.5.4 Matting Despills Together
      4. 5.4 The Unspill Operation
        1. 5.4.1 How to Set It Up
        2. 5.4.2 Grading to the Backing Color
      5. 5.5 Despill Artifacts
        1. 5.5.1 Finding the Artifacts
        2. 5.5.2 Hue Shifts
        3. 5.5.3 Dark Edges
        4. 5.5.4 Fixing Despill Artifacts
      6. 5.6 Edge Grading
      7. 5.7 Edge Extension
    5. Chapter 6 – The Composite
      1. 6.1 Premultiply vs. Unpremultiply
        1. 6.1.1 Premultiply
        2. 6.1.2 Unpremultiply
        3. 6.1.3 The Double Premultiply
      2. 6.2 The Composite
        1. 6.2.1 The Over Composite
        2. 6.2.2 The KeyMix Composite
        3. 6.2.3 The AddMix Composite
          1. 6.2.3.1 How It Works
          2. 6.2.3.2 How to Build It
          3. 6.2.3.3 How to Use It
        4. 6.2.4 The Processed Foreground Method
          1. 6.2.4.1 The Workflow
          2. 6.2.4.2 What to Watch Out For
      3. 6.3 Compositing With a Keyer
        1. 6.3.1 Soft Comp/Hard Comp
        2. 6.3.2 “Cut and Paste” Keyer Compositing
      4. 6.4 Compositing Outside the Keyer
        1. 6.4.1 The Single Key
        2. 6.4.2 The Uberkey
        3. 6.4.3 Soft Key/Hard Key
        4. 6.4.4 The Additive Keyer
      5. 6.5 Stereo Compositing
        1. 6.5.1 Anaglyph
        2. 6.5.2 Stereopsis
        3. 6.5.3 Stereoscopy
        4. 6.5.4 The Stereo Conversion Process
        5. 6.5.5 Depth Grading
          1. 6.5.5.1 Scene Transition
          2. 6.5.5.2 The Dashboard Effect
          3. 6.5.5.3 Window Violation
          4. 6.5.5.4 Miniaturization
          5. 6.5.5.5 Divergence
        6. 6.5.6 Stereo Compositing
          1. 6.5.6.1 Dual View Display
          2. 6.5.6.2 Split and Join Views
          3. 6.5.6.3 Disparity Maps
    6. Chapter 7 – Compositing CGI
      1. 7.1 Multi-pass CGI Compositing
        1. 7.1.1 Process Verification for Your Renderer
        2. 7.1.2 Render Passes
        3. 7.1.3 Lighting Passes
          1. 7.1.3.1 Render Passes Workflow
          2. 7.1.3.2 Beauty Pass Workflow
        4. 7.1.4 AOVs
        5. 7.1.5 ID Passes
        6. 7.1.6 Normals Relighting
      2. 7.2 EXR File Format
        1. 7.2.1 Film Scans
        2. 7.2.2 Linear Lightspace
        3. 7.2.3 Arbitrary Image Channels
      3. 7.3 HDR Images
      4. 7.4 Deep Compositing
        1. 7.4.1 Deep Images
        2. 7.4.2 The Layering Complexity Problem
        3. 7.4.3 The Depth Compositing Edge Problem
        4. 7.4.4 The Re-rendering Problem
        5. 7.4.5 Deep Compositing with Live Action
    7. Chapter 8 – 3D Compositing
      1. 8.1 A Short Course in 3D
        1. 8.1.1 The 3D Coordinate System
        2. 8.1.2 Vertices
        3. 8.1.3 Meshes
        4. 8.1.4 Surface Normals
        5. 8.1.5 UV Coordinates
        6. 8.1.6 Map Projection
        7. 8.1.7 UV Projection
        8. 8.1.8 3D Geometry
        9. 8.1.9 Geometric Transformations
        10. 8.1.10 Geometric Deformations
          1. 8.1.10.1 Image Displacement
          2. 8.1.10.2 Noise Displacement
          3. 8.1.10.3 Deformation Lattice
        11. 8.1.11 Point Clouds
        12. 8.1.12 Lights
        13. 8.1.13 Shaders
        14. 8.1.14 Reflection Mapping
        15. 8.1.15 Ray Tracing
        16. 8.1.16 Image-based Lighting
        17. 8.1.17 Cameras
      2. 8.2 3D Compositing
        1. 8.2.1 3D Compositing from 2D Images
        2. 8.2.2 Pan and Tile
        3. 8.2.3 Camera Projection
        4. 8.2.4 Multiplane Shots
        5. 8.2.5 Set Extension
        6. 8.2.6 3D Backgrounds
      3. 8.3 Alembic Geometry
        1. 8.3.1 The Simple Case
        2. 8.3.2 Scenegraphs
        3. 8.3.3 Advantages Over FBX
      4. 8.4 Camera Tracking
        1. 8.4.1 Step 1 – Feature Tracking
        2. 8.4.2 Step 2 – The Solve
        3. 8.4.3 Step 3 – Build the Scene
        4. 8.4.4 Placing the Geometry
        5. 8.4.5 A Large Outdoor Scene
  10. PART II THE QUEST FOR REALISM
    1. Chapter 9 – Color Correction
      1. 9.1 The Behavior of Light
        1. 9.1.1 The Inverse Square Law
        2. 9.1.2 Diffuse Reflections
        3. 9.1.3 Specular Reflections
        4. 9.1.4 Bounce Light
        5. 9.1.5 Scattering
      2. 9.2 Gamma
        1. 9.2.1 The Math
        2. 9.2.2 Why Do We Need Gamma?
      3. 9.3 The Affect of Color Operations
        1. 9.3.1 Lift
        2. 9.3.2 Gamma
        3. 9.3.3 Gain
        4. 9.3.4 Offset
        5. 9.3.5 Saturation
        6. 9.3.6 Color Grading vs. Color Correcting
        7. 9.3.7 Increasing Contrast with the “S” Curve
        8. 9.3.8 Histograms
        9. 9.3.9 Channel Swapping
        10. 9.3.10 Premultiply vs. Unpremultiply – Again
      4. 9.4 Matching the Light Space
        1. 9.4.1 Brightness and Contrast
          1. 9.4.1.1 Matching the Black and White Points
          2. 9.4.1.2 Matching the Midtones
          3. 9.4.1.3 Gamma Slamming
        2. 9.4.2 Matching Color
          1. 9.4.2.1 Grayscale Balancing
          2. 9.4.2.2 Flesh Tones
          3. 9.4.2.3 The “Constant Green” Method of Color Correction
          4. 9.4.2.4 Daylight
          5. 9.4.2.5 Specular Highlights
        3. 9.4.3 Lighting Direction
        4. 9.4.4 Quality of Light Sources
          1. 9.4.4.1 Creating Softer Lighting
          2. 9.4.4.2 Creating Harsher Lighting
        5. 9.4.5 Non-linear Gradients for Color Correction
        6. 9.4.6 The DI Process
        7. 9.4.7 A Checklist
    2. Chapter 10 – Sweetening the Comp
      1. 10.1 Layer Integration
      2. 10.2 Interactive Lighting
      3. 10.3 Edge Blending
      4. 10.4 Light Wrap
      5. 10.5 Creating Shadows
        1. 10.5.1 Edge Characteristics
        2. 10.5.2 Density
        3. 10.5.3 Color
        4. 10.5.4 Faux Shadows
        5. 10.5.5 Shadow Warping
        6. 10.5.6 Contact Shadows
      6. 10.6 Atmospheric Haze
      7. 10.7 Adding a Glow
      8. 10.8 Grain Management
        1. 10.8.1 Grain Characteristics
        2. 10.8.2 Regraining Techniques
          1. 10.8.2.1 Regrain Tool
          2. 10.8.2.2 Lifted Grain
          3. 10.8.2.3 Grain Rescue
        3. 10.8.3 Grain Management Workflows
          1. 10.8.3.1 Live Over Live
          2. 10.8.3.2 Live Over CGI
          3. 10.8.3.3 CGI Over Live
          4. 10.8.3.4 CGI Over CGI
          5. 10.8.3.5 Still Photos
      9. 10.9 Managing Clipping
    3. Chapter 11 – Camera Effects
      1. 11.1 Lens Effects
        1. 11.1.1 Lens Distortion
        2. 11.1.2 Depth of Field
        3. 11.1.3 Vignetting
        4. 11.1.4 Lens Defects
          1. 11.1.4.1 Spherical Aberration
          2. 11.1.4.2 Astigmatism
          3. 11.1.4.3 Chromatic Aberration
        5. 11.1.5 Glows and Flares
          1. 11.1.5.1 Lens Flare
          2. 11.1.5.2 Lens Filter Flare
          3. 11.1.5.3 Diffraction Glows
          4. 11.1.5.4 Veiling Glare
        6. 11.1.6 Grain
      2. 11.2 Lens Distortion Workflows
        1. 11.2.1 CGI Over Live Action
        2. 11.2.2 Live Action Over CGI
        3. 11.2.3 CGI Over CGI
        4. 11.2.4 Live Action Over Live Action
      3. 11.3 Matching the Focus
        1. 11.3.1 Using a Blur for Defocus
        2. 11.3.2 How to Simulate a Defocus
        3. 11.3.3 Sharpening
          1. 11.3.3.1 Sharpening Operations
          2. 11.3.3.2 Unsharp Masks
          3. 11.3.3.3 Making Your Own Unsharp Mask
      4. 11.4 Rolling Shutter
  11. PART III THINGS YOU SHOULD KNOW
    1. Chapter 12 – Digital Color
      1. 12.1 Color Spaces
        1. 12.1.1 Primary Chromaticities
        2. 12.1.2 Units of Measure
        3. 12.1.3 Transfer Function
        4. 12.1.4 Gamut
        5. 12.1.5 HSV and HSL
        6. 12.1.6 Log and Linear
      2. 12.2 Working in Linear
        1. 12.2.1 What Exactly Is Linear?
        2. 12.2.2 Color Operations
        3. 12.2.3 Transformations and Filtering
        4. 12.2.4 CGI
      3. 12.3 Metadata
      4. 12.4 OpenColorIO
      5. 12.5 ACES Color Management
        1. 12.5.1 The ACES Workflow
        2. 12.5.2 The ACES Gamut
        3. 12.5.3 What About Video Productions?
    2. Chapter 13 – Image Blending
      1. 13.1 Image Blending in Linear Light Space
        1. 13.1.1 Image-blending Operations
        2. 13.1.2 Compositing Operations
        3. 13.1.3 Matching the Look of sRGB in Linear
          1. 13.1.3.1 All sRGB Color Space
          2. 13.1.3.2 sRGB Within Linear
      2. 13.2 Alpha Compositing Operations
      3. 13.3 Image-blending Operations
        1. 13.3.1 The Screen Operation
          1. 13.3.1.1 Adjusting the Appearance
        2. 13.3.2 The Weighted Screen Operation
        3. 13.3.3 Multiply
          1. 13.3.3.1 Adjusting the Appearance
        4. 13.3.4 Maximum
        5. 13.3.5 Minimum
        6. 13.3.6 Absolute Difference
      4. 13.4 Adobe Photoshop Blending Modes
        1. 13.4.1 Simple Blending Modes
        2. 13.4.2 Complex Blending Modes
      5. 13.5 Slot Gags
      6. 13.6 Retiming Clips
        1. 13.6.1 Constant Speed Changes
        2. 13.6.2 Variable Speed Changes
        3. 13.6.3 Interpolation Methods
          1. 13.6.3.1 Nearest Neighbor
          2. 13.6.3.2 Frame Average
          3. 13.6.3.3 Motion Estimation
      7. 13.7 VR Stitching
        1. 13.7.1 Workflow Overview
        2. 13.7.2 Removing Lens Distortion
        3. 13.7.3 Building a Matching Computer Rig
        4. 13.7.4 Projecting Onto the Panosphere
        5. 13.7.5 The Stitching Process
        6. 13.7.6 Coping with Parallax
        7. 13.7.7 Exposure Correction
        8. 13.7.8 Visual Effects
    3. Chapter 14 – Transforms and Tracking
      1. 14.1 Geometric Transforms
        1. 14.1.1 2D Transforms
          1. 14.1.1.1 Translation
          2. 14.1.1.2 Rotation
          3. 14.1.1.3 Resize vs. Scale
          4. 14.1.1.4 Skew
          5. 14.1.1.5 Corner Pinning
        2. 14.1.2 Managing Motion Blur
          1. 14.1.2.1 Transform Motion Blur
          2. 14.1.2.2 Motion UV Motion Blur
          3. 14.1.2.3 Speed Changes
        3. 14.1.3 3D Transforms
        4. 14.1.4 Filtering
          1. 14.1.4.1 The Effects of Filtering
          2. 14.1.4.2 Twinkling Starfields
          3. 14.1.4.3 Choosing a Filter
        5. 14.1.5 Lining Up Images
          1. 14.1.5.1 Offset Mask Lineup Display
          2. 14.1.5.2 Edge-detection Lineup Display
          3. 14.1.5.3 The Pivot Point Lineup Procedure
      2. 14.2 Image Displacement
      3. 14.3 Warps and Morphs
        1. 14.3.1 Mesh Warps
        2. 14.3.2 Spline Warps
        3. 14.3.3 Morphs
        4. 14.3.4 Tips, Tricks, and Techniques
      4. 14.4 Point Tracking
        1. 14.4.1 The Tracking Operation
          1. 14.4.1.1 Selecting Good Tracking Targets
          2. 14.4.1.2 Bad Tracking Targets
          3. 14.4.1.3 Tracker Enable/Disable
          4. 14.4.1.4 Offset Tracking
          5. 14.4.1.5 Keep Shape and Follow Shape
          6. 14.4.1.6 Pre-processing the Clip
          7. 14.4.1.7 Coping with Grain
          8. 14.4.1.8 Tracking Workflow
          9. 14.4.1.9 Cleaning up Tracking Data
          10. 14.4.1.10 The Stability Test
          11. 14.4.1.11 Reasons for Failure
        2. 14.4.2 Match Move
          1. 14.4.2.1 2D Transforms
          2. 14.4.2.2 Corner Pinning
        3. 14.4.3 Stabilizing
          1. 14.4.3.1 The Repo Problem
          2. 14.4.3.2 Motion Smoothing
          3. 14.4.3.3 Stabilizing For Rotoscoping
      5. 14.5 Planar Tracking
        1. 14.5.1 The Planar Grid
        2. 14.5.2 Drift Correction
        3. 14.5.3 Exporting Data
        4. 14.5.4 Roto Assist
    4. Chapter 15 – Digital Images
      1. 15.1 HD Video
        1. 15.1.1 Frame Formats
        2. 15.1.2 Anamorphic Video
        3. 15.1.3 Scan Modes
        4. 15.1.4 Working with Interlaced Video
          1. 15.1.4.1 De-interlacing
          2. 15.1.4.2 Scan Line Interpolation
          3. 15.1.4.3 Field Averaging
        5. 15.1.5 Color Subsampling
        6. 15.1.6 Keying with 4:2:2 Video
        7. 15.1.7 Frame Rates
          1. 15.1.7.1 24, 25, 30, 60fps
          2. 15.1.7.2 23.98, 29.97, 59.94fps
        8. 15.1.8 Timecodes
        9. 15.1.9 Video File Formats
        10. 15.1.10 Telecine
          1. 15.1.10.1 The 3:2 Pull-down
          2. 15.1.10.2 The 3:2 Pull-up
      2. 15.2 Digital Cinema Images
        1. 15.2.1 Digital Camera Advantages
        2. 15.2.2 The Bayer Array
        3. 15.2.3 Sensor Crop
        4. 15.2.4 HFR – High Frame Rate
        5. 15.2.5 The DCI
      3. 15.3 Film Scans
        1. 15.3.1 Grain
        2. 15.3.2 The “Safe-to” Window
        3. 15.3.3 Apertures
        4. 15.3.4 Aspect Ratios
        5. 15.3.5 Film Formats
          1. 15.3.5.1 Full Aperture
          2. 15.3.5.2 Academy Aperture
          3. 15.3.5.3 Super 35 Formats
          4. 15.3.5.4 Cinemascope
          5. 15.3.5.5 Working with Cscope
          6. 15.3.5.6 “3-perf” Film
          7. 15.3.5.7 VistaVision
          8. 15.3.5.8 65mm/70mm
          9. 15.3.5.9 IMAX
      4. 15.4 Log Images
        1. 15.4.1 What Are Log Images?
        2. 15.4.2 Why We Need Log Images
          1. 15.4.2.1 Human Vision
          2. 15.4.2.2 Data Compression
          3. 15.4.2.3 Working with Log Images
      5. 15.5 Light Field Cinematography
        1. 15.5.1 How It Works
        2. 15.5.2 The Impact on Visual Effects
          1. 15.5.2.1 Deep Images
          2. 15.5.2.2 Arbitrary Depth of Field
          3. 15.5.2.3 Depth Maps
          4. 15.5.2.4 Stereo Through a Single Lens
          5. 15.5.2.5 Volumetric Optical Flow
          6. 15.5.2.6 Position Pass
          7. 15.5.2.7 Point Clouds
          8. 15.5.2.8 Mattes
          9. 15.5.2.9 Normals and Normal Relighting
          10. 15.5.2.10 Camera Tracking
        3. 15.5.3 When, and How Much?
  12. Glossary
  13. Index

Product information

  • Title: Digital Compositing for Film and Video, 4th Edition
  • Author(s): Steve Wright
  • Release date: November 2017
  • Publisher(s): Routledge
  • ISBN: 9781315283999