Scala and Spark for Big Data Analytics

Dataset collection and exploration

In this section, we will describe the very famous MNIST dataset. This dataset will be used throughout this chapter. The MNIST database of handwritten digits (downloaded from https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/multiclass.html) has a training set of 60,000 examples and a test set of 10,000 examples. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image. Consequently, this is a very good example dataset for those who are trying to learn techniques and pattern recognition methods on real-world data while spending minimal efforts on preprocessing and formatting. The original black and white (bi-level) images from NIST were ...