Building Machine Learning Systems with Python - Third Edition

Looking under the hood

How does jug work? At the basic level, it's very simple. Task is a function plus its argument. Its arguments may be either values or other tasks. If a task takes other tasks, there is a dependency between the two tasks (and the second one cannot be run until the results of the first task are available).

Based on this, jug recursively computes a hash for each task. This hash value encodes the whole computation to get the result. When you run jug execute, for each task, there is a little loop that runs the logic, depicted in the following flowchart:

The default backend writes the file to disk (in this funny folder named ...