Frank Kane's Taming Big Data with Apache Spark and Python

Running the code

So let's go ahead and run that and see it in action. Open up Canopy Command Prompt and run the following:

 spark-submit word-count-better-sorted.py

These results should look a little bit better, a little bit more useful, a little more interesting:

We have now sorted by the number of times each word occurred, and the most popular word in my book is the word you. Words, To, your, the, a, of, and, that are the most popular-all the words you'd expect to show up a lot in any book really. When we start looking at more interesting words, business turns out to be the most popular, which makes sense because it is a book about starting ...