Scala Machine Learning Projects

Step 7 - Setting up an optimizer

As suggested by many researchers, the RMSProp optimizer helps an LSTM network to converge quickly. Therefore, I have decided to use it here too:

val opt = new RMSProp(learningRate = learningRate) 

Additionally, the model parameters to be optimized are its parameters, except the training data and the label (weights and biases):

val paramBlocks = model.symbol.listArguments() 
      .filter(x => x != "data" && x != "softmax_label") 
      .zipWithIndex.map { case (name, idx) => 
        val state = opt.createState(idx, model.argsDict(name)) 
        (idx, model.argsDict(name), model.gradDict(name), state, name) 
      }    .toArray