Hands-On Deep Learning with Apache Spark

RNNs with DL4J and Spark

The example presented in this section is an LSTM that would be trained to generate text, one character at a time. The training is done using Spark.

The dependencies for this example are the following:

• Scala 2.11.8
• DL4J NN 0.9.1
• ND4J Native 0.9.1 and the specific classifier for the OS of the machine where you would run it
• ND4J jblas 0.4-rc3.6
• Apache Spark Core 2.11, release 2.2.1
• DL4J Spark 2.11, release 0.9.1_spark_2

We start configuring the network as usual through the NeuralNetConfiguration.Builder class, as follows:

val rng = new Random(12345)val lstmLayerSize: Int = 200val tbpttLength: Int = 50val nSamplesToGenerate: Int = 4val nCharactersToSample: Int = 300val generationInitialization: String = nullval conf ...