Introducing Data Science

Chapter 5. First steps in big data

This chapter covers

• Taking your first steps with two big data applications: Hadoop and Spark
• Using Python to write big data jobs
• Building an interactive dashboard that connects to data stored in a big data database

Over the last two chapters, we’ve steadily increased the size of the data. In chapter 3 we worked with data sets that could fit into the main memory of a computer. Chapter 4 introduced techniques to deal with data sets that were too large to fit in memory but could still be processed on a single computer. In this chapter you’ll learn to work with technologies that can handle data that’s so large a single node (computer) no longer suffices. In fact it may not even fit on a hundred computers. ...