Multithreading for Visual Effects

Chapter 2

Houdini: Multithreading Existing Software

Jeff Lait

Side Effects Software, Inc.

2.1 What Is Houdini? . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . 19

2.2 Rewrite or Refactor . . . . . . .. . . . . . . . . . .. . . . . . . . .. .. . . . . . . . .. . . . . . . . . . .. . . . . . . . . . . 21

2.2.1 Cleaning Statics . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . .. . . . . . . . . . .. . . 22

2.2.2 Threading the Simple Cases . . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . .. 27

2.3 Patterns . .. . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . .. .. . . . . . . . .. . . . . . . . 30

2.3.1 Always Be Reentrant . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . .. . . . . . . . . . .. . . . . . 30

2.3.2 Never Lock . . . . . . .. . . . . . . . . . .. . . . . . . . .. .. . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. 31

2.3.3 Atomics Are Slow . . . . . .. . . . . . . . . . .. . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . 31

2.3.4 Never Blindly Thread . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . 32

2.3.5 Command Line Control . . . . . . .. . . . . . . . . . .. . . . . . . . .. .. . . . . . . . .. . . . . . . . . 33

2.3.6 Constant Memory versus Number of Cores . . . . . . . . .. . . . . . . . . . .. . . . . . . 33

2.3.7 Memory Allocation . . . . . .. . . . . . . . . . .. . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . 34

2.4 Copy on Write . . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. 34

2.4.1 Const Correctness . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . .. . . . . . . . . . .. . . . . . 35

2.4.2 Reader/Writer Locks . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . .. .. . . . . . . . .. . . . . . 35

2.4.3 Ownership Is Important . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . .. .. . . . . . . . 36

2.4.4 Sole Ownership Is a Writer Lock . . . .. . . . . . . . . . .. . . . . . . . .. .. . . . . . . . .. . . 37

2.4.5 Failure Modes of This System . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . 38

2.5 Dependencies . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . .. .. . . . . . . . .. . . . . . . . . . 40

2.5.1 Task Locks . . . . . . . . . .. . . . . . . . . . .. . . . . . . . .. .. . . . . . . . .. . . . . . . . . . .. . . . . . . . . 41

2.5.2 Mantra . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . .. . . 43

2.6 OpenCL . . . . . . . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . .. .. . . . . . . . .. . . . . . . . . . .. . . 44

2.1 What Is Houdini?

In this chapter we share the experiences and pitfalls of our ongoing quest to make

Houdini fully multithreaded. We will attempt to make the examples as context-agnostic

as possible, but we will no doubt still fall into some specialized jargon. In this section we

provide the missing context.

Houdini is the flagship package of Side Effects Software, http://www.sidefx.com. Fig-

ure 2.1 shows a screenshot of a simulation inside Houdini. It is a complete 3D animation

and effects package and is well known for its extremely procedural approach to art cre-

ation. Our focus on proceduralism stems from our goal to create tools for artists to express

themselves with computers. Computer generated art is distinct from prior mediums in its

proceduralism: computers excel at repeating rote tasks. Unfortunately, the act of instruct-

ing computers is considered a highly technical task, and often seen as divorced from the act

19