Errata

Something is wrong with the last equation, h(t) = (1 - z(t)) ¤ tanh (WxgT * h(t-1) + z(t)¤gt)

I think it should be: h(t) = (1 - z(t)) ¤ h(t-1) + z(t) ¤ g(t)

Note from the Author or Editor:
Yes indeed, you are absolutely right, I don't know what I was thinking when I wrote this equation, I apologize. The correct equation to compute h(t) is, as you wrote:

h(t) = (1 - z(t)) ⊗ h(t-1) + z(t) ⊗ g(t)

The equation in latexmath format is:
\mathbf{h}_{(t)}&=(1-\mathbf{z}_{(t)}) \otimes \mathbf{h}_{(t-1)} + \mathbf{z}_{(t)} \otimes \mathbf{g}_{(t)}

Thanks again for contributing to improving this book, I hope you are enjoying it.

Regarding the L0 norm, the text says "L0 just gives the cardinality of the vector (i.e., the number of elements)...". It may be clearer if the text says: "the number of non-zero elements"

Note from the Author or Editor:
Good point, thanks. I actually fixed this a few weeks ago. The online version and the latest printed copies should be fixed by now. The sentence is now:

"ℓ0 just gives the number of non-zero elements in the vector, and ℓ∞ gives the maximum absolute value in the vector."

On Figure 2-13 the axis values and legend is not shown. This due to a bug in #Matplotib inline on "scatter" plots. The attributes are hidden. Bellow you can find a temporary solution using sharex=False to restore visibility. The comment line cites the source for the solution.

housing2.plot(kind = "scatter", x = "longitude", y = "latitude", alpha = 0.4, s = housing2["population"]/100,
label = "population", c = "median_house_value", cmap = plt.get_cmap("jet"), sharex=False)
# sharex=False fixes a bug. Temporary solution. See: https://github.com/pandas-dev/pandas/issues/10611

Note from the Author or Editor:
I just love it when people come with both the problem and the solution! :)
I just tried your bug fix and it works fine, thanks a lot.

At least on my system (Ubuntu/Kubuntu), pip3 install --user installs virtualenv command in ~/.local/bin, which is not in my PATH. Calling virtualenv provokes a response that the user should install it using sudo apt-get install virtualenv. Doing that leads to problems with mixing versions. So a note on adding ~/.local/bin to the PATH could be useful for inexperienced python programmers like myself --- both in the book and on the github page. BTW, you complained in the errata that \mathbf{\theta} did not work. It should be \bm{\theta}.

Note from the Author or Editor:
Hi Jan,
Thanks for your feedback. I'm sorry you had trouble with the installation instructions: I actually hesitated to add any installation instructions to my book, because it's really the sort of things that varies a lot across systems, and changes over time. I'll add a footnote as you suggest, it's a great idea.

Regarding the bold font theta, someone suggested using \bm instead of \mathbf a while ago, and I tried, but it did not work. For example, go to latex2png.com and try running "x \mathbf{x} \bf{x} \theta \mathbf{\theta} \bf {\theta}". I see a normal x, then 2 identical bold x, then 3 identical normal thetas. O'Reilly ended up converting many of the equations to MathML, and then it worked fine.
Cheers,
Aurélien

The text states "slightly over 800 districts have a median_house_value equal to about $500,000.

I suppose you meant "slightly over 1,000 districts", looking at the peak in the relevant histogram (the x-axis numbers overlap in the book, but it's the lonely peak at the right).

Unless you consider 1,000+ to also be "slightly over 800" :)

Note from the Author or Editor:
Good catch, thanks! I actually meant to write "equal to about $100,000".

there's a minor error on page 50 that produces a bug for python version < 3

This line has the hash() function that returns the ascii code number in python 3:
return hash(np.int64(identifier)).digest()[-1] < 256 * test_ratio

in python version 2 this returns a character which breaks the entire function.

Replacing the line with this fixes it:

if sys.version[0] == '3':
return hash(np.int64(identifier)).digest()[-1] < 256 * test_ratio
else:
return ord(hash(np.int64(identifier)).digest()[-1]) < 256 * test_ratio

Cheers

Note from the Author or Editor:
Thanks for your feedback. Indeed, this function only works with Python 3.

In the notebook, I proposed a version that supports both Python 2 and 3:

def test_set_check(identifier, test_ratio, hash):
return bytearray(hash(np.int64(identifier)).digest())[-1] < 256 * test_ratio

It's kind of ugly, so I decided to just present the Python 3 version, but I should have added a comment to make it clear.

Side note: most scientific python libraries have announced that they will stop supporting Python 2 very shortly (e.g., NumPy will stop releasing new features in Python 2 at the end of this year, see https://python3statement.org/ for more details).

One problem with the implementation above is that it uses the MD5 hash and only looks at a single byte, so the cut between train and test is rather coarse. Since then, I found a better option using CRC32 (much faster and returning 4 bytes, so it's much more fine grained), which I will be proposing in future releases:

from zlib import crc32

def test_set_check(identifier, test_ratio):
return crc32(np.int64(identifier)) & 0xffffffff < test_ratio * 2**32

def split_train_test_by_id(data, test_ratio, id_column):
ids = data[id_column]
in_test_set = ids.apply(lambda id_: test_set_check(id_, test_ratio))
return data.loc[~in_test_set], data.loc[in_test_set]

This works just as well on Python 2 and Python 3 (in Python 3, you could remove "& 0xffffffff", which is only needed because crc32() returns a signed int32 in Python 2, while it is unsigned int32 in Python 3).

Hope this helps!
Aurélien

(1st Edition)

Last sentence of the paragraph below a code block.
"The names can be anything you like."

But actually step name can't include double underscore(__). :-)

Note from the Author or Editor:
Indeed, the only constraint is that it should not contain double underscores, thanks for pointing it out.

The text reads: "Standardization is quite different: first it subtracts the mean value (so standardized values always have a zero mean), and then it divides by the variance so that the resulting distribution has unit variance." I think that it should be "..it divides by the standard deviation..". According to the StandardScaler source code it also divides by the standard deviation and not the variance.

Note from the Author or Editor:
Good catch! Of course you are right, first subtract the mean, then divide by the standard deviation, not the variance. Thanks a lot.

room_ix should be rooms_ix

- bedrooms_per_room = X[:, bedrooms_ix] / X[:, room_ix]
----
+ bedrooms_per_room = X[:, bedrooms_ix] / X[:, rooms_ix]

Note from the Author or Editor:
Great catch. Thanks. This error is now fixed.

Best regards,
Aurélien

I believe that the denominator in the equation below is incorrect. Should be dividing by households rather than population.

ERROR -> housing["rooms_per_household"] = housing["total_rooms"]/housing["population"]

CORRECT -> housing["rooms_per_household"] = housing["total_rooms"]/housing["households"]

Note from the Author or Editor:
Thanks a lot for your feedback. I fixed the error, it will disappear from the electronic versions shortly, and the printed copy will not contain it.

Best regards,
Aurélien

From Scikit-learn 0.18, train_test_split is included in sklearn.model_selection.

Note from the Author or Editor:
Thanks a lot for your feedback. This error is now fixed.

Best regards,
Aurélien

>>> precision_score(y_train_5, y_pred)

Should be

>>> precision_score(y_train_5, y_train_pred)

Note from the Author or Editor:
Good catch, thanks. I tested every line of code before adding it to the book, but I guess I must have renamed this variable in the notebook at one point, and when I updated the book I missed a couple occurrences. Sorry about that!

Note that there's the same problem a few lines below:

>>> f1_score(y_train_5, y_pred)

should be:

>>> f1_score(y_train_5, y_train_pred)

I fixed these issues, but it may take a while for them to propagate to the digital version.

The published book states "More specifically, this is a multivariate regression problem since the system will use multiple features to make a prediction (it will use the district’s population, the median income, etc.). In the first chapter, you predicted life satisfaction based on just one feature, the GDP per capita, so it was a univariate regression problem.”

Instead of multivariate regression, it should be multiple regression. Multivariate regression is where there is more than 1 dependent variable, while multiple regression refers to more than 1 predictor/independent variable - which is this case.

Note from the Author or Editor:
Oops, indeed you are right, I should have said "multiple", not "multivariate", I just fixed this.
Thanks!

plt.legend(loc="bottom right")

should be

plt.legend(loc="lower right")

Note from the Author or Editor:
Good catch, thanks.

>>> sgd_clf.classes[5]

should be

>>> sgd_clf.classes_[5]

Note from the Author or Editor:
Good catch, thanks.

The last sample code on page 100,
>>> y_train_knn_pred = cross_val_predict(knn_clf, X_train, y_train, cv=3)
>>> f1_score(y_train, y_train_knn_pre, average='marco'),

may be corrected to
>>> y_train_knn_pred = cross_val_predict(knn_clf, X_train, y_multilabel, cv=3)
>>> f1_score(y_multilabel, y_train_knn_pred, average='marco').

Note from the Author or Editor:
Thanks for your feedback, you are absolutely right (with one minor tweak: it's "macro", not "marco"). I updated the book and the jupyter notebook.

Cheers,
Aurélien

In the code example, the random noise generated for the training set is overwritten with noise for the test set before it is applied:

noise = rnd.randint(0, 100, (len(X_train), 784))
noise = rnd.randint(0, 100, (len(X_test), 784))
X_train_mod = X_train + noise
X_test_mod = X_test + noise

Just switch the second and third line to get it right (as it is in the notebook on github):

noise = rnd.randint(0, 100, (len(X_train), 784))
X_train_mod = X_train + noise
noise = rnd.randint(0, 100, (len(X_test), 784))
X_test_mod = X_test + noise

Note from the Author or Editor:
Good catch, thanks! Indeed, it should be written as you indicate, just like in the notebook. I'm not sure how this error happened. I fixed it now (but it may take a while to propagate to the digital versions).

The normal equation is stated to determine the parameters of the linear regression model:
\theta = (X^T X)^{-1} X^T y
Later, the computational complexity of calculating the inverse is mentioned. But, there is an alternative not mentioned in the book. One can determine \theta as the solution of the linear equation
(X^T X) \theta – X^T y = 0

This should be explained in the book, as well. Later, in the section “Linear regression with TensorFlow” the same mistake is made. In case this is on purpose, because the calculation shows well how to use TensorFlow, you should at least mention it.

Best regards and thank you for the great book,
Niclas

Note from the Author or Editor:
Indeed, you are right, the Normal Equation is not the only way to determine the parameters of the Linear Regression model. I updated the book to also mention the alternative you propose, which leads to using the Moore-Penrose pseudoinverse of X. This in turn requires computing the SVD of X (see chapter 8 for the SVD). This solution is both faster to compute, and it supports collinear data (e.g., a dataset where one or more features are linear combinations of other features), which the normal equation does not support. This is what Scikit-Learn actually uses. Thanks for your suggestion!

You refer to the process of gradually reducing the learning rate as simulated annealing.

Other sources use this term to refer to an algorithm that occasionally makes "uphill" moves (with a probability decreasing over time).

I see the analogy, but I think you're using the terminology in a nonstandard way here.

Note from the Author or Editor:
Thanks for your feedback. Indeed, it's an analogy, not an identity. I updated the sentence like so:

This process is akin to simulated annealing, an algorithm inspired from the process of annealing in metallurgy where molten metal is slowly cooled down.

For a more detailed explanation of the link between gradient descent using a learning schedule and simulated annealing, see:
http://leon.bottou.org/publications/pdf/nimes-1991.pdf

- >>> some_data_prepared = preparation_pipeline.transform(some_data)
------
+ >>> some_data_prepared = full_pipeline.transform(some_data)

Note from the Author or Editor:
Thanks for your feedback. This error is now fixed.

Best regards,
Aurélien

svm_clf.fit(X_scaled)

should be

svm_clf.fit(X)

The pipeline is performing the scaling and there is no "X_scaled" variable elsewhere in the sample.

Note from the Author or Editor:
Good catch, thanks. Indeed, it should be:

svm_clf.fit(X)

rather than:

svm_clf.fit(X_scaled)

I tested every code example in the book, but it seems that a few times I updated the notebooks and forgot to update the book. I just wrote a script to compare the code in the notebooks with the code examples in the book, and I'm currently going through every chapter to fix the little differences. This is one of them.

This code returns an error:
iris = datasets.load_iris()
X=iris["data"][:,(2,3)] #only petal length and width
y=(iris["target"]==2).astype(np.float64) #import only IrisVirginica

svm_clf = Pipeline((
("scaler", StandardScaler()),
("linear_svc", LinearSVC(C=1,loss="hinge")),
))

svm_clf.fit(X,y)


error:
~\Miniconda3\envs\MyEnv\lib\site-packages\sklearn\pipeline.py in _fit(self, X, y, **fit_params)
224 # transformer. This is necessary when loading the transformer
225 # from the cache.
--> 226 self.steps[step_idx] = (name, fitted_transformer)
227 if self._final_estimator is None:
228 return Xt, {}

TypeError: 'tuple' object does not support item assignment

Note from the Author or Editor:
Thanks for your feedback.
The code actually works fine up to Scikit-Learn 0.18, but then in Scikit-Learn 0.19 (which did not exist when I wrote the book), Pipelines must now be created with a list of tuples instead of a tuple of tuples. I updated the Jupyter notebooks to ensure that the code now works with Scikit-Learn 0.19. Basically, use this code instead (note the square brackets):

svm_clf = Pipeline([
("scaler", StandardScaler()),
("linear_svc", LinearSVC(C=1,loss="hinge")),
])


Cheers,
Aurélien

Code at the bottom of the page should include the call to make_moons(); otherwise X and y will be fit from the iris dataset when working through the chapter in order.

Note from the Author or Editor:
Good catch, thanks!
Indeed, the following line was missing in the code:

X, y = make_moons(n_samples=100, noise=0.15)

I just fixed this. Thanks again.

The value of vector b is supposed to be -1 I think. Since you introduced -1*t to A, if the vector of b is made of 1s then the whole formula after substitution will be
t(wx+b) >= -1, according to my calculation.


And I really think some part of the dot product and matrix-vector multiplication is messed up. Is this convention in machine learning to use dot product represent matrix multiplication?

Note from the Author or Editor:
Great catch! The vector b should be full of -1 instead of 1.
The constraints are defined as: p^T a^(i) <= b^(i), for i=1, 2, ..., m
If b^(i) = -1, we can rewrite the constraints as: p^T a^(i) <= -1
Since a^(i) = -t^(i) x^(i), the constraints are: -t^(i) p^T x^(i) <= -1
Which we can rewrite to: t^(i) p^T x^(i) >= 1
For positive instances, t^(i) = +1, and for negative instances t^(i) = -1.
So for positive instances: p^T x^(i) >= 1, which is what we want.
For negative instances: -p^T x^(i) >= 1, therefore: p^T x^(i) <= -1, which is also what we want.
Thanks a lot for your feedback, I fixed the error for the next release.

I believe here the linear transformation and dot product is mistaken from a math perspective. It's supposed to be a dot b not a transpose dot b.

Note from the Author or Editor:
Thanks for your feedback. Indeed, it's probably better to replace `a^T b` with `a.b` in this section. In many cases in Machine Learning, it's more convenient to represent vectors as column vectors (i.e., 2D arrays with a single column), so they can be transposed, used like matrices, and so on. Of course if `a` and `b` are column vectors, then `a^T b` is a 2D array containing a single cell whose value is equal to the dot product of the (1D) vectors corresponding to `a` and `b`. In other words, the result is identical, except for the dimensionality: if `a` and `b` are regular vectors, then `a.b` is a scalar, but if `a` and `b` are column vectors, then `a^T b` is one-cell matrix. For example:

>>> import numpy as np
>>> np.array([2,3]).dot(np.array([5,7])) # a.b
31
>>> np.array([[2],[3]]).T.dot(np.array([[5],[7]])) # a^T b
array([[31]])

I plan to cleanup the whole book regarding this issue, not just chapter 5 (but it may take a bit of time).

Thanks again!

Original phrase

Page 188: Chapter 7: Ensemble Learning and Random Forests

"has a 60.6% probability of belonging to the positive class (and 39.4% of belonging to the positive class):"

There are the word "positive class" two times. If 39.4% is the probability to be in the positive class, I think 100 - 39.4% which is 60.6 should be the probability to be in the negative class.

Which number is for negative and which one is for positive class, then? Please help, thank you.

Note from the Author or Editor:
Good catch, thanks! Indeed, the sentence should be:

"""
For example, the oob evaluation estimates that the first training instance has a 68.25% probability of belonging to the positive class (and 31.75% of belonging to the negative class):
"""

(1st Edition)

In Equation 8-1, V^T should be V.
This is often confused, because svd() function actually returns V^T, not V

So, I suggest to change code below Eq 8-1
U, s, V = np.linalg.svd(X_centered)
c1 = V.T[:, 0]
c2 = V.T[:, 1]
-->
U, s, Vt = np.linalg.svd(X_centered)
c1 = Vt.T[:, 0]
c2 = Vt.T[:, 1]

In next page(p314) first sentence,
"the maxtix composed of the first d columns of V^T"
should be
"the maxtix composed of the first d columns of V"

Thanks

Note from the Author or Editor:
Good catch, thanks!

Here is the list of changes I just made, which includes your list plus a couple more fixes:
* Top of page 213, "where V^T contains all the principal components" was changed to "where V contains all the principal components".
* Equation 8-1: replaced V^T with V.
* In all code examples, replace V with Vt. This includes 3 replacements in the code on page 213, and 1 replacement in the first code example on page 214.
* Top of page 214: "the matrix composed of the first d columns of V^T" was changed to "the matrix composed of the first d columns of V".

I also updated the corresponding notebook, and added a comment to explain the issue.

Thanks again! :)

You have "y depends on w, which depends on x".

I believe y depends on x, which depends on w.

Note from the Author or Editor:
Good catch, thanks again Peter. :)
Indeed the sentence should read:

TensorFlow automatically detects that y depends on x, which depends on w, so it first evaluates w, then x, then y, and returns the value of y.

tf.global_variable_initializers()

should be

tf.global_variables_initializer()

Note from the Author or Editor:
Great catch, thanks! This error is now fixed, it was a failed find&replace, when the method `initialize_all_variables()` got renamed to `global_variables_initializer()`.

Best regards,
Aurélien

"Zeta is the 8th letter of the Greek alphabet"

It is the 6th letter of the Greek alphabet.

Note from the Author or Editor:
Indeed, Zeta is the 6th letter of the Greek alphabet, thanks!

Code example:

>>>dnn_clf.evaluate(X_test,y_test)

Doesn't supported

should be

>>>dnn_clf.score(X_test,y_test)

instead.

Note from the Author or Editor:
Thanks for your feedback. The code works fine in TensorFlow 1.0, but it breaks in TensorFlow 1.1, because TF.Learn's API was changed significantly. I noticed this a while ago and I updated the book accordingly (I removed the paragraph about evalution because TF.Learn seems to be a moving target), so this problem only affects people who have the first revision of the book and are using TF 1.1+.
Cheers,
Aurélien

The assertion
"the function is smooth everywhere, including around z = 0"
is only true if alpha = 1.

Note from the Author or Editor:
Good point, you are absolutely right. I corrected this sentence like this:

Third, if alpha is equal to 1 then the function is smooth everywhere, including around z = 0, which helps speed up Gradient Descent, since it does not bounce as much left and right of z = 0.

Thanks for your feedback!

Your example how the Nesterov update is converging faster is wrong in my opinion. The example is a function with one variable (teta). In consequence, the gradient (and the momentum, too) is one-dimensional in each point. But you draw them as 2-dim tangential vectors, which leads in your wrong assumption that the Nesterov update is going closer to the optimum in this example.

Why is your assumption wrong:

You can clearly see in the graph that
- eta * gradient1 > - eta * gradient2 > 0

and
beta * m > 0. (looking at the x-component)

This leads to
beta * m - eta gradient1 > beta * m - eta gradient2 > 0

and
|beta * m - eta gradient1| > |beta * m - eta gradient2| > 0

which is a clear contradiction to your drawing.

There are real examples when the Nesterov update is better than the regular momentum update:
- It is crossing a local minimum / stationary point faster
- If the regular momentum update goes farther than the optimum, the Nesterov update does not go as far away from the optimum (in some situations).

#stilllovingyourbook

Note from the Author or Editor:
Excellent catch, thanks! I tried to fix the figure while keeping the cost function 1D, but it looked bad, and it didn't make Nesterov Accelerated Gradient seem very useful at all, so I ended up changing the figure altogether to make the cost function 2D. Hopefully it should be in the tenth release of the 1st edition (which should come out very shortly, in December 2018).
Thanks again!

For further callback method names, looks like "on_epoch_begin()" is there twice, but no "..._end". Same for "on_batch_end()" where there's no "_begin". A copy/paste mixup?

Note from the Author or Editor:
Great catch, thanks!

The sentence should be:

As you might expect, you can implement `on_train_begin()`, `on_train_end()`, `on_epoch_begin()`, `on_epoch_end()`, `on_batch_begin()` and `on_batch_end()`.

Page number of error is not exact since I have the kindle (azw) version.

The error is at Chapter 10. Perceptron learning rule of Equation 10-2.

W(next step) = W + eta(y_hat - y)x # (estimation - true_label)
should be
W(next step) = W + eta(y - y_hat)x # (true_label - estimation)

Note from the Author or Editor:
Good catch, indeed this is a mistake. Equation 10-2 should have target - estimation rather than estimation - target. In latex math, the equation should be:

{w_{i,j}}^{(\text{next step})} = w_{i,j} + \eta (y_j - \hat{y}_j) x_i

rather than:

{w_{i,j}}^{(\text{next step})} = w_{i,j} + \eta (\hat{y}_j - y_j) x_i

Thank you!

In the second line of the code, it should call q.enqueue_many() instead of q.enqueue()

So, that line should be:

enqueue_many = q.enqueue_many([training_instances])

Note from the Author or Editor:
Good catch! The text says you can use "enqueue_many" and then I use "enqueue" in the code example, I was probably out of coffee. ;-) That line of code should be:

enqueue_many = q.enqueue_many([training_instances])

Thanks a lot,
Aurélien

he means to define the line filters = np.zeros(shape=(7, 7, channels, 2), dtype=np.float32) , but he calls the variable filters_test, like the two lines below it . The jupyter notebook doesn't make that mistake, though

Note from the Author or Editor:
Good catch, thanks! I probably renamed the variable at one point and missed a few occurrences, sorry about that.

This is now fixed, but it may take some time to propagate to production.

Text says
------------
Specifically, a neuron located in row i, column j of the feature map k in a given convolutional layer l is connected to the outputs of the neurons in the previous layer l – 1, located in rows i × sw to i × sw + fw – 1 and columns j × sh to j × sh + fh – 1, across all feature maps (in layer l – 1).

Concern
-----------
The book defines sw as horizontal stride, and sh as vertical stride. Cool.

My intuition is that the horizontal stride changes the feature map's number of columns. And vertical stride changes the the feature map's number of rows.

Should it be:
a) the horizontal stride sw (not sh) should affect the column ranges?
b) the vertical stride sh (not sw) should affect the row ranges?


Correction
--------------
Specifically, a neuron located in row i, column j of the feature map k in a given convolutional layer l is connected to the outputs of the neurons in the previous layer l – 1, located in rows i × sh to i × sh + fw – 1 and columns j × sw to j × sw + fh – 1, across all feature maps (in layer l – 1).


Please forgive me if I'm wrong. Just plodding through the book and doing 'back of the envelope' calculations/exercises as I go.

Regards,
dre

Note from the Author or Editor:
Good catch, this is indeed an error, my apologies. Moreover, it helped me find an error in Equation 13-1. I double-checked the rest of pages 357-361 and they seem fine to me.

The sentence at the bottom of page 361 should be:

Specifically, a neuron located in row i, column j of the feature map k in a given convolutional layer l is connected to the outputs of the neurons in the previous layer l - 1, located in rows i x sh to i x sh + fh - 1 and columns j x sw to j x sw + fw - 1, across all feature maps (in layer l - 1).

The Equation 13-1 should be (using latexmath):

z_{i,j,k} = b_k + \sum\limits_{u = 0}^{f_h - 1} \, \, \sum\limits_{v = 0}^{f_w - 1} \, \, \sum\limits_{k' = 0}^{f_{n'} - 1} \, \, x_{i', j', k'} . w_{u, v, k', k}
\quad \text{with }
\begin{cases}
i' = i \times s_h + u \\
j' = j \times s_w + v
\end{cases}

The difference is that u, v and k' must be zero-indexed, and i'=i x sh + u instead of i'=u x sh + fh - 1, and similarly j' = j x sw + v instead of j' = v x sw + fw - 1.

You can view the updated equation (and all equations in the book) at:
http://nbviewer.jupyter.org/github/ageron/handson-ml/blob/master/book_equations.ipynb

Thank you very much for your help,
Aurélien Géron

Eq 14-1 implies:
---------------------
The value y(t) is a vector quantity.


Concern
------------
The way I understand Figure 14-2 is that each neuron in the recurrant layer outputs a single scalar value per timestep, and these scalars make up the vector quantity y.

Specifically, each element of the vector y comes from only one of the neuron's output in the recurrant layer.

But single neuron equation 14-1 implies that y(t)) is a vector quantity.

Dimensional analysis of Eq 14-1 requires the value of y(t) to be a scalar if:

1. bias b is a scalar and
2. x(t) and w_x and y_t-1 and w_y are vectors


Eq 14-1 Correction
-------------------------

y(t) should not be bold face, implying that it's a scalar quantity specific to one neuron.

Note from the Author or Editor:
Once again, good catch! My intention was actually to show the equation for a whole recurrent layer on a single instance (i.e., on one input sequence), not for a single neuron. So the equation is correct but the title is wrong. It should have been:

Equation 14-1. Output of a recurrent layer for a single instance

I will also fix the sentence introducing this equation, replacing "single recurrent neuron" with "recurrent layer":

The output of a recurrent layer can be computed pretty much as you might expect, as shown in Equation 14-1 [...]

Thanks for your very helpful feeback,
Aurélien Géron

Text says
-------------
b is a vector of size n_neurons containing each neuron's bias term.

Concern
------------
Using this definition of b, the only way to properly add all the terms inside Eq 14-2 is to broadcast the bias term. Otherwise we're adding terms of different shapes.

The text does not mention this explicitly and can be confusing if you don't know what's 'going on under the hood', i.e broadcasting of b.

Let's assume the shape of the bias term is (1, n_neurons), therefore having size of n_neurons. In Eq14-2 (the first line), the other two product terms inside the activation function result in a shape of:

1. Shape of X_(t) . W_x is = (m, n_neurons)
2. Shape of Y_(t-1) . W_y is = (m, n_neurons)

Which leads to requiring the bias term to also be of shape (m, n_neurons), so we broadcast m times along b's first dimension.

(This broadcasted shape of b also works in the second line of Eq 14-2.)

Correction
--------------
Maybe mention that the bias is being broadcasted (for those of us who are unfamiliar with it), or otherwise change the definition of its shape to be (m, n_neurons)?

Note from the Author or Editor:
That's a great point. In fact, I should have mentioned this earlier, in chapter 10, the first time we use broadcasting when adding a bias vector. I just added the following sentence at the end of point 5 at the bottom of page 266:

Note that adding a 1D array (*b*) to a 2D matrix with the same number of columns (*X* . *W*) results in adding the 1D array to every row in the matrix: this is called _broadcasting_.

Thanks a lot,
Aurélien Géron

The problem I observed was actually with the Jupyter notebook "14_recurrent_neural_networks.ipynb" currently (2017 July 13) on GitHub -- but the particular code with the problem is associated approximately with the text on page 386 of the printed book (illustrating the "static_rnn()" function).

Specifically, the output of "In [14]:" (show_graph(tf.get_default_graph())), which is supposed to be a graph of some kind, is instead a big empty space (1200 px X 620 px).

Similarly, the output of "In [26]:", in code demonstrating the result of "dynamic_rnn()", is also a big blank space.

Looking at the Firefox web-developer "Console" window, I see two JavaScript logging items which seem to say that "HTML Sanitizer" has changed the "iframe.srcdoc" value from what appears to be meaningful data to "null". Specifically, code in "/notebook/js/main.min.js" seems to be the place doing the sanitizing.

Configuration: Windows 7 64-bit, Firefox 48.0, Anaconda3 version 4.4.0 (2017-05-11), Python 3.6.1, Jupyter 5.0.0, TensorFlow 1.2.1. So, some of the package versions are later than the book, but I think the issue here is worth investigating.

Aside: This particular notebook ("14_recurrent_neural_networks.ipynb") currently contains a few more minor problems: "In [69]:", "In [77]:", and "In [103]", all call functions which begin with "rnd". However, while it seems previous versions of the notebook included a statement "import numpy.random as rnd", the code has evidently been changed so that "rnd" is no longer defined. Changing the three instances of "rnd" to "numpy.random" fixes all three problems -- enabling the entire notebook to be executed in Jupyter (but the problem mentioned at the top of this note, namely the blank graph areas, remains; but does not cause the notebook to stall execution midway, perhaps because the operation succeeded but was "sanitized" away).

Note from the Author or Editor:
Thanks for your feedback. I just fixed the `rnd` issue in the Jupyter notebook, and I pushed the updated notebook to github (FYI, I use these imports so often that I added them to my python startup script, which is why I was not getting any error).

Regarding the `show_graph()` function, it does not seem to work across all browsers, unfortunately. I use Chrome, and the graph is displayed just fine, but some people have reported that it fails on Firefox, indeed. I'll try to find a way to make it work in Firefox, but in the meantime, the official way to visualize a TensorFlow graph is to use TensorBoard (see chapter 9).

The equation for h_t appears to be incorrect. Instead of
h_t = (1 - z_t) * h_(t - 1) + z_t * g_t
the Cho et al. (2014) paper has in equation 7
h_t = z_t * h_(t - 1) + (1 - z_t) * g_t
Accordingly, the “1-” unit in figure 14-14 on p. 406 should be moved right, to the path leading from z_t to the multiplication with the output of g_t. (And the label for the g_t is missing.)
Your visualizations of the RNN cells are a great help, and are much appreciated!

Note from the Author or Editor:
Thanks for your feedback. You are right that my graph & equations inverted z_t and 1 - z_t. Fortunately, the GRU cell works fine either way. Indeed, the z gate is trying to learn the right balance between forgetting old memories (let's call this f) and storing new ones (let's call this i). In a GRU cell, f = 1 - i. In the paper, the z gate outputs f, while in my book, it outputs i. Either way, the right balance will be found just as well.
If you want an analogy, it's as if you were learning how empty a glass should be, while I was learning how full it should be. The net result is the same, but somehow I find the latter a bit more natural. ;) That said, even though "my" equations will work fine, I will fix them so that people don't get confused when they see other implementations or read the paper.

The code shows the creation of "your first environment" and reads:

>>> import gym
>>> env = gym.make("CartPole-v0")
[2016-10-14 16:03:23,199] Making new env: Ms Pacman-v0
[,...]

the output code was probably copied from further onto the chapter, since it should be (and I quote my own output)

[2017-09-13 10:48:27,402] Making new env: CartPole-v0

Note from the Author or Editor:
Nice catch, thanks! I fixed this, the next digital and paper editions should be good.

In Chpater 16,
Discount rate(r) is different discount factor(\gamma).
Discount factor \gamma = 1/(1+r).

So I recommend:
'discount rate' in text should be 'discount factor'.
'discount_rate' in code should be 'discount_factor'.

Thanks.

Note from the Author or Editor:
Thanks Haesun, that's a good point. I used "discount rate" to mean "discount factor", and I have seen several people do the same, but you are right that it's clearer to replace "discount rate" with "discount factor" everywhere in chapter 16. I just did this. In the code examples, I have a constraint of using 80 characters max per line, so I cannot easily replace discount_rate with discount_factor, so instead I replaced discount_rate with gamma, with a comment in the code every time I define gamma, for example:
gamma = 0.95 # the discount factor
Also, the first time that the discount factor is introduced (just before figure 16-6), instead of naming it "r", I named it gamma. This avoids possible confusion with rewards (which are named "r") later in the chapter, and it also makes the chapter more consistent.
Thanks for your suggestion!

Labels (a1, a2, s2, s3) in the text for Figure16-8 are incorrectly printed.

Note from the Author or Editor:
Thanks for your feedback. I'm not sure exactly what you mean by "printed incorrectly". Are you referring to the text font (I am not seeing a problem)? Or to the fact that the text contained a couple errors (e.g., inverted a1 and a2, and s2 and s3)? I assume it's the latter. I fixed these errors:

BEFORE: In state _s_~1~ it has only two possible actions: _a_~0~ or _a_~1~. It can choose to stay put by repeatedly choosing action _a_~1~, or it can choose to move on to state _s_~2~ and get a negative reward of -50 (ouch). In state _s_~3~ it has no other choice [...] and in state _s_~3~ the agent has no choice but to take action [...].

AFTER: In state _s_~1~ it has only two possible actions: _a_~0~ or _a_~2~. It can choose to stay put by repeatedly choosing action _a_~0~, or it can choose to move on to state _s_~2~ and get a negative reward of -50 (ouch). In state _s_~2~ it has no other choice [...] and in state _s_~2~ the agent has no choice but to take action [...]

Thanks again!
Aurélien

The learning_rate is defined as 0.01 but it is never used. It is not a real problem as the algorithm doesn't take a learning rate. But it is confusing when you read the code.

Note from the Author or Editor:
Indeed, the learning_rate is unused in this code, I just removed it.

Thanks!
Aurélien Géron

Comparing the contents of the array R in the Python code to Figure 16-8, I believe the second occurrence of the value -10.0 in the definition of R should be 0.0, in this code:

R = np.array([ # shape=[s, a, s']
:
[[10., 0.0, 0.0], [nan, nan, nan], [0.0, 0.0, -50.]],
:
])

There is no transition from state S1 through action a0 back to state S0, so a reward of +10 does not do anything here.

It also does no harm, but it may reduce some confusion :)

Note from the Author or Editor:
Ha! Good catch! :) Indeed, that line should read:

[[0.0, 0.0, 0.0], [nan, nan, nan], [0.0, 0.0, -50.]],

As you point out, this is a reward for a transition that has 0% probability, so it doesn't change the result, but I agree that it's potentially confusing. I've fixed it in the book (the notebook was already okay, somehow, I must have noticed the issue in the notebook at some point, but forgot to fix it in the book).

Thanks for your help! :)

I've downloaded the latest version of the PDF and ePub from my OReilly account.

PDF is version is 2017-06-09.
ePub is version 2017-06-09

Concern
------------
The code for Q-Learning seems to not match Equation 16-5. In particular, how the learning rate (aka alpha) is used.

Currently code reads:

[...]
Q[s, a] = learning_rate * Q[s, a] + (1 - learning_rate) * (
reward + discount_rate * np.max(Q[sp])
)
[...]


To agree with Equation 16-5 it should be:

[...]
Q[s, a] = (1 - learning_rate) * Q[s, a] + learning_rate* (
reward + discount_rate * np.max(Q[sp])
)
[...]

I've checked though the Jupyter notebook for the Reinforcement chapter and it looks to agree with Equation 16-5, albeit the code is setup a little different.


Aside
--------
It looks like this latest version of the PDF, ePub doesn't have some of the corrections previously fixed.

e.g. The description for the states in Figure 16-7 p456 PDF still refer to the nonexistent state s3. I haven't checked any other fixed errata but could it be that O'Reilly have not correctly setup/linked to the newest up-to-date version? Just adding another 'data point' to hopefully help if it's confusing others.

Or I'm doing something wrong. I don't know.

Almost at the end :) Thanks again for the book, stuff is finally 'clicking'.

Note from the Author or Editor:
Good catch, thanks! Indeed, the code was wrong, it should have been as you said, reversing (1 - learning_rate) and learning_rate, just like in Equation 16-5. I just pushed the fix to O'Reilly's git repo, so both the digital editions and new printed books should be fixed within the next couple of weeks.

Regarding the description of Figure 16-7, it is normal that it mentions state s3 since that state exists on the figure. However, if you see state s3 still mentioned in the description of Figure 16-8, then there's a problem. I'll contact O'Reilly to to make 100% sure that all the digital editions are up to date.

Note: I have sync'ed the code examples from all chapters with the code in the Jupyter notebooks, except for chapters 15 and 16, which are not 100% synchronized yet.

The answer to the second part of question 2 in Chapter 13: Convolutional Neural Networks reads:

"...this first layer takes up 4 x 100 x 150 x 100 = 6 million bytes (about 5.7 MB)...The second layer takes up 4 x 50 x 75 x 200 = 3 million bytes (about 2.9 MB). Finally, the third layer takes up 4 x 25 x 38 x 400 - 1,520,000 bytes (about 1.4 MB). However, once a layer has been computed, the memory occupied by the previous layer can be released, so if everything is well optimized, only 6 + 9 = 15 billion bytes (about 14.3 MB) of RAM will be required (when the second layer has just been computed, but the memory occupied by the first layer is not released yet)."

For the situation described, if both the first and second layers are in memory, would that not be 3 + 6 = 9 million bytes (8.58 MB) of RAM required? When you add the amount occupied by the CNN's parameters (3,613,600 bytes) that would be a total of about 12 MB for predicting a single instance.

I could also be missing something really obvious so sorry if that is the case. Either way, thanks for the great, enjoyable book!

Note from the Author or Editor:
You are correct, I have no idea why I wrote 6+9 instead of 6+3. Thanks a lot!

I just fixed the paragraph like this:

"""
However, once a layer has been computed, the memory occupied by the previous layer can be released, so if everything is well optimized, only 6 + 3 = 9 million bytes (about 8.6 MB) of RAM will be required (when the second layer has just been computed, but the memory occupied by the first layer is not released yet). But wait, you also need to add the memory occupied by the CNN's parameters. We computed earlier that it has 903,400 parameters, each using up 4 bytes, so this adds 3,613,600 bytes (about 3.4 MB). The total RAM required is (at least) 12,613,600 bytes (about 12.0 MB).
"""