Errata

Note Fastai has changed so this should be
head = create_head(512*2,2,ps=0.5) not
head = create_head(512*4,2,ps=0.5) and revise text.

Note from the Author or Editor:
Indeed, this should be changed as suggested.

while we are discussing how to discriminate '3' and '7' in this chapter, this paragraph is talking about '8'.
I guess '8' here should be '3'.

and the syntax of the function for probability of being the number 8 is strange.

def pr_eight(x, w) = (x*w).sum()

it should be something like,

def pr_three(x, w): return (x*w).sum()

Note from the Author or Editor:
For consistency with the previous page, I guess we can use 3s here, so replace all 8 by 3 (I counted five instances on this page) and pr_eight by pr_three.

Here, SymPy has taken the derivative of x**2 for us!

'x' should be 'sx',

Here, SymPy has taken the derivative of sx**2 for us!

Note from the Author or Editor:
Yes, the sentence should be fixed as proposed.

"To do the dot product of our weight matrix (2 by number of activations) with the
activations (batch size by activations by rows by cols), we use a custom einsum":

the activation does not include the batch size.

Note from the Author or Editor:
Replace "(batch size by activations by rows by cols)" by "(batch size by rows by cols)"

x.shape

I cannot understand why we check the shape of x here.
I guess it should be

act.shape

Note from the Author or Editor:
p521 replace

x.shape
torch.Size([1, 3, 224, 224])

by

act.shape
torch.Size([512, 7, 7])

"Before we do, we’ll call a hook, if it’s defined. "

'Before' should be 'After', according to the code above.

# I assume that by 'do' the authors mean 'call forward'.

Note from the Author or Editor:
replace the sentence "Before we do, we'll call a hook, if it's defined" by "After, we call a hook, if it's defined".

"except in camel_case."

'camel_case' should be 'snake_case'

Note from the Author or Editor:
Indeed, replace 'camel_case' with 'snake_case' in the second to last paragraph.

xxunk
Indicates the next word is unknown


I understand this special token means the word is unknown. not the next word.

Note from the Author or Editor:
Yes, replace "Indicates the next word is unknown" by "Indicates this word is unknown"

"So, we want to transform our numbers between 0 and 1 to instead be between negative infinity and infinity. "

logarithm does not give us infinity for 1. it should be 'negative infinity and 0' .

Note from the Author or Editor:
Indeed, "negative infinity and infinity" should be replaced by "negative infinity and 0"

"To decide if an output represents a 3 or a 7, we can just check whether it's greater than 0."

I understand that the threshold here is 0.5, so 0 should be 0.5.

Also, the code snippet just below this paragraph
----
(preds>0.0).float() == train_y[:4]
----
0.0 should be 0.5

Note from the Author or Editor:
Yes, in the paragraph mentioned, 0 should be replaced with 0.5 and in the code snippet immediately below, 0.0 should be 0.5.

tensor([1,2,3]) + tensor([1,1,1])

this does not make sense as an example.

i guess it should be

tensor([1,2,3]) + tensor(1)

Note from the Author or Editor:
Yes, this should be `tensor([1,2,3]) + tensor(1)`

"For the gradients of the ReLU and our linear layer, we use the gradients of the loss with respect to the output (in out.g) and apply the chain rule to compute the gradients of the loss with respect to the output (in inp.g)."

the last 'the gradients of the loss with respect to the output (in inp.g)' is doubtful.
'output' should be 'input', or 'output of the previous layer'?

Note from the Author or Editor:
Replace
"For the gradients of the ReLU and our linear layer, we use the gradients of the loss with respect to the output (in out.g) and apply the chain rule to compute the gradients of the loss with respect to the output (in inp.g)."
by
"For the gradients of the ReLU and our linear layer, we use the gradients of the loss with respect to the output (in out.g) and apply the chain rule to compute the gradients of the loss with respect to the input (in inp.g)."

"the scale of our activations will go from 1 to 0.1, and after 100 layers"

the follwing code uses 50, not 100.

Note from the Author or Editor:
Replace 100 by 50

"Scale (1d tensor): (1) 256 x 256"

it is 2d tensor, not 1d, i guess.

Note from the Author or Editor:
In the last paragraph of the page, replace 1d in "Scaled (1d tensor): (1) 256 x 256" by 2d.

CancelFitException and CancelBatchException

the label and explanation do not match.

CancelFitException and CancelBatchException
should be exchanged.

Note from the Author or Editor:
In p487-488, switch the labels CancelFitException and CancelBatchException.

"You can pass `emb_drop` to `__init__` to change this value:"

i could not find init parameter emb_drop forTabularModel
https://github.com/fastai/fastai/blob/master/fastai/tabular/model.py

This should be "You can pass `embd_p` to `__init__` to change the dropout probabilty:" ?

Note from the Author or Editor:
Replace `emb_drop` by `embed_p` on p467, in the sentence "you can pass emb_drop to __init__ to change this value"

"where conv is the function from the previous chapter that adds a second convolution, then a ReLU, then a batchnorm layer"

the conv in p.437 uses batchnorm *before* ReLU.

Note from the Author or Editor:
On p447 replace ""where conv is the function from the previous chapter that adds a second convolution, then a ReLU, then a batchnorm layer" by "where conv is the function from the previous chapter that adds a second convolution, then a batchnorm layer, then a ReLU"

"Why do we scale the weights with dropout?"
I understand weights are not scaled. 'activations' are scaled.

Note from the Author or Editor:
In q33 replace "weights" by "activations"

" and passing x[0].isupper(), which evaluates to True if the first letter is uppercase (i.e., it’s a cat)."

The code is not passing 'x[0].isupper()', instead it passes a fucntion 'is_cat'

Note from the Author or Editor:
replace 'x[0].isupper()', by 'is_cat' in this sentence.

In the 3rd line of the 3rd paragraph. it should be " for a total of 784 pixels" not "768 pixels" ]. Change 768 to 784

Note from the Author or Editor:
Please replace 768 by 784 as advised.

The class SiameseImage is derived from Tuple (note the capital T). The native Python datatype is written all in lower case 'tuple' but I think what's meant here is fastuple, the extended tuple in fastcore. I wonder if Tuple was renamed fastuple at some point.

Note from the Author or Editor:
Please replace all instance of "Tuple" (with the capital, code-formatted) by "fastuple" (I counted two on p364, one in the code of the class SiameseImage, one in the paragraph before and one on p365).

[channels_in, features_out, rows, columns] should be read:
[features_out, channels_in, rows, columns]
Indeed, a few lines later, the shape of the kernel is shown to be [4,1,3,3], which corresponds to my suggested correction, but not to the original print.
Or am I missing something?

Note from the Author or Editor:
Please make the modification proposed by the reader.

"we'll get back 1,010 absolute values" should read
"we'll get back 1,010x28x28 absolute values" -
(valid_3_tens-mean3).abs().shape -> torch.Size([1010, 28, 28])

Note from the Author or Editor:
change to "get back 1,010 matrices of absolute values"