Apache Spark Deep Learning Cookbook

How it works...

1. The array function will be used to handle data in the form of arrays. The numpy library provides this function readily.
2. Since our data is only text data, we will require the string library to handle all input data as strings before encoding the words as integers, which can be fed.
3. The tokenizer function will be used to split all the sentences into tokens, where each token represents a word.
4. The pickle library will be required in order to save the dictionary into a pickle file by using the dump function.
5. The to_categorical function from the keras library converts a class vector (integers) to a binary class matrix, for example, for use with categorical_crossentropy, which we will require at a later stage in order to map tokens ...