Errata

Contrary to what is stated in the book, R does not appear to come pre-installed on Mac OS X.

At least, I can't find it, and the instructions as given do not work. (i.e. typing R in terminal gives 'command not found')

Perhaps the author had already installed R?

Checking a dozen or so of the top sites for "r mac os x" shows they all refer to installing R. I didn't find any references to R being preinstalled.

The URLs for ggplot2 and glmnet are reversed.

The location for the tm package is shown as http://www.spatstat.org/spatstat/, this however is the location for the SpatStat package. The location should be: http://tm.r-forge.r-project.org/index.html

The package_installer.R script is not in "the code folder for this chapter". It is in the root directory of the example code. There is no "code" directory in chapter 1.

Also, there does not appear to be a link to the example code in the book. It's easily found on the website, but it should be in the book. (and it should be in the index of the book).

And the script has no error checking, so even if it fails on the first package, it keeps running, and failing, all the way through.

And it fails as it requires the mac developer tools to run (for make, etc). These are mentioned as optional on page 7. They are apparently required.

In addition, page 7 also says "requires both the C and Fortran compilers... you can install these compilers using the mac os x developers tools DVD". I don't have the DVD, but the downloaded version should be identical, and does not contain Fortran. This must be installed separately. (http://cran.r-project.org/bin/macosx/tools/). This is fairly severe, as the install will continue to the end, but the error message scroll past, unless you are watching, leaving you with a broken install. This seems likely to confuse new R users

And still some errors. Will report them later, after the build finishes.

Most or all of this could be avoided if the install script used binary packages, not source.

The data file ufo_awesome.tsv is way too big for quick processing. I'm trying out the statements in R while reading the book. The statement "ufo<-read.delim(...)" practically froze my Mac because of this large data file. Maybe provide a smaller file for quick programming along.

The number of bad rows reads "371" instead of "731". Here is the printout when the code is run:

> good.rows<-ifelse(nchar(ufo$DateOccurred)!=8 | nchar(ufo$DateReported)!=8,FALSE,TRUE)
> length(which(!good.rows))
[1] 731

The strsplit function doesn't throw an error when the split character isn't matched -- it just returns the string, so the [[1]] reference apparently will always return something.

Perhaps the solution here is to reference [[2]] rather than [[1]], to check if a split occurred?

seems like the assignment to ufo$USState should be
ufo$USState <- ufo$USState[...]

I think you have a ggplot version issue. When trying to generate the histogram for UFO sightings, I get the following error:

"Error in continuous_scale(aesthetics, "date", identity, breaks = breaks, :
unused argument(s) (major = "50 years")

I am using R 2.14.2 and ggplot 0.9.0.

It would appear someone on Stack Exchange is having the same error: http://stackoverflow.com/questions/9857123/error-in-continuous-scale-and-error-in-inherits-ggplot2-r-2-14-2

Here you discuss Fig. 2-11 and the weights of women and men.
Therefore, instead of the curve centers 64" and 69" (inches), the means of the weights in pounds should be given.

For the scale_x_date function, the code in the book uses "major" as the parameter name, when it should be "breaks" e.g. scale_x_date(breaks = "5 years", ...
The same applies for the other uses of scale_x_date in this chapter. The example code is accurate.

When I create the plot on page 71 with the code on page 70, the "Height" and "Weight" axes are switched.

You discuss blue dots and red dots in Fig. 3-1.
This figure is monochrome and displays circles and triangles.

the code/data folder...
should be replaced by
the data folder...

The code to generate the picture that is distributed in the book should be updated to correspond to the latest version of ggplot. Specifically, in the file email_classify.R,

ex1 <- ggplot(val, aes(x, V2)) +
geom_jitter(aes(shape = as.factor(V3)),
position = position_jitter(height = 2)) +
scale_shape_discrete(legend = FALSE, solid = FALSE) +
geom_hline(aes(yintercept = c(10,30), linetype = 2)) +
theme_bw() +
xlab("X") +
ylab("Y")

should be:
ex1 <- ggplot(val, aes(x, V2)) +
geom_jitter(aes(shape = as.factor(V3)),
position = position_jitter(height = 2)) +
scale_shape_discrete(guide = "none", solid = FALSE) +
geom_hline(aes(yintercept = c(10,30) )) +
theme_bw() +
xlab("X") +
ylab("Y")

This repairs two errors: "legend" is deprecated, and a new error: "A continuous variable can not be mapped to linetype" that is generated by "linetype = 2". Note that removing linetype = 2 produces solid, not dashed lines. This is consistent with the book text, but no longer matches the corresponding figure in the book.

This function generates the indicated error message:

all.spam <- sapply(spam.docs, function(p) get.msg(paste(spam.path,p,sep="/")))
Error in seq.default(which(text == "")[1] + 1, length(text), 1) :
wrong sign in 'by' argument

it also generates warning messages on several spam files, the first such file being: "data/spam/00006.5ab5620d3d7c6c0db76234556a16f6c1". The error comes from the line:

?We've received 8,000 in 1 day and we are doing

and is generated because of the first character, which appears to be outside the usual ascii range.

The culprit is in the following function:

get.msg <- function(path) {
con <- file(path, open="rt", encoding="latin1")
text <- readLines(con)
# The message always begins after the first full line break
msg <- text[seq(which(text=="")[1]+1,length(text),1)]
close(con)
return(paste(msg, collapse="\n"))
}

If "latin1" is changed to "native.enc", then this error stops, and the all.spam <- sapply(spam.docs, function(p) get.msg(paste(spam.path,p,sep="/")))
works.

Note that there is a second typo in the above line as well, reported in a separate errata. (sep = "/" not "")

all.spam <- sapply(spam.docs, function(p) get.msg(paste(spam.path,p,sep="")))

should read:

all.spam <- sapply(spam.docs, function(p) get.msg(paste(spam.path,p,sep="/")))

The stopwords are still showing up in spam.df. Someone else also posted this in December. Any news?

In recent versions of the tm package minDocFreq=2 has been replaced by bounds = list(global = c(2,Inf)). See https://stackoverflow.com/questions/16287546/trying-to-remove-words-from-a-documenttermmatrix-in-order-to-use-topicmodels

In classify.email R rounds to zero the product of probabilities of a long term lists.

I solved this issue using log transformation:

ClasifyEmail <- function(path, training.df, prior=0.5, c=1e-6) {
msg <- GetMsg(path)
msg.tdm <- GetTDM(msg)
msg.matrix <- as.matrix(msg.tdm)
msg.freq <- rowSums(msg.matrix)
# Find intersection of words
msg.match <- intersect(names(msg.freq), training.df$term)
# Compute probabilities of the unseen terms
unseen.probs <- prior*c^(length((msg.freq))-length(msg.match))
unseen.probs.log <- log(prior)+(length((msg.freq))-length(msg.match))*log(c)
if (length(msg.match) < 1) {
return(unseen.probs)
} else {
# Search matched terms probs
match.probs <- training.df$occurrence[match(msg.match,training.df$term)]
# Compute probability of occurrence ot the terms
# Add probabilities of the unseen terms
prob <- unseen.probs*prod(match.probs)
prob.log <- unseen.probs.log + sum(log(match.probs))
return (prob.log)
}
}

replace sep="" in two places in this code with sep="/"

easyham.parse <- lapply(easyham.docs, function(p) parse.email(paste(easyham.path, p, sep="")))

should read:

easyham.parse <- lapply(easyham.docs, function(p) parse.email(paste(easyham.path, p, sep="/")))

The get.date function fails because the second line of the function says (note the 'l' after 'grep'):

date.grepl <- which(date.grep == TRUE)

and the third line says (note the missing 'l'):

date <- msg.vec[date.grep[1]]


The sample code uses date.grep for both the second and third line.

As with the error reported for Page 80, the encoding should be "native.enc", not "latin1", otherwise this generates an error message.

It occurs to me that this errata, as with the page 80 one, may only apply on one type of machine, as other machines may not generate this error, and "latin1" may be correct there. I generated this error on a Windows 7 64 bit computer.

R^2 is calculated as 1 - (model.rmse / mean.rmse), but these values should be MSEs rather than RMSEs.

Source: http://en.wikipedia.org/wiki/Coefficient_of_determination defines R^2 using MSEs (it doesn't actually take means, but the divisions cancel); and the R^2 reported by `summary(fitted.regression)` gives the same value as calculating it using MSEs but not RMSEs.

The code in the 2nd sentence of the 2nd paragraph "sqrt(mean(residuals(lm.fit) ^ 2))" should be replaced by "sqrt((sum(residuals(lm.fit) ^ 2)) / 998)". The Residual Standard Error does not strictly use the mean of the squared residuals but rather the sum of the squared residuals divided by n - p (in this case 998), where p is the number of predictors in your model including intercept.

"In this example, the a parameter is the slope of the line and the b parameter is the intercept" disagrees with the preceding code snippet and following paragraphs. "a" and "b" need swapping for it to be correct.

While reading ch.8, page 207, I wondered why the percentages of variance added up to more than 100%.

I later found time to check with some experts:
http://stats.stackexchange.com/q/32901/5503

and got confirmation that the text is incorrect. The quoted paragraph
could be changed to:

In this summary, the standard deviations tell us how much of the
variance in the data set is accounted for by the different principal
components. Use summary(pca) to see the proportions of variance. The
first component, called Comp.1, accounts for 46% of the variance, while
the next component accounts for another 22.7%. By the end, the last
component, Comp.24, accounts for a mere 0.01% of the variance. This
suggests that we can learn a lot about our data by just looking at the
first principal component.

First two lines of code should read:

comparison <- transform(comparison, MarketIndex = scale(MarketIndex))
comparison <- transform(comparison, DJI = scale(DJI))

sep="" in the second line on the page should be sep="/" (at least on the Windows 7 machine on which I am testing)

URLs should be split by a slash, not a backslash as stated. The code listing is correct, but the text is not.