This step builds the most fundamental part of our program—the neural network graph.
First, we start by initializing the hyperparameters of the model, as shown in the following example:
learning_rate = 0.001batch_size = 512number_of_iterations = 100000number_hidden_units = 1024
One often experiments with the above values until the model receives decent results:
- The learning_rate (https://towardsdatascience.com/understanding-learning-rates-and-how-it-improves-performance-in-deep-learning-d0d4059c1c10) is used in backpropagation and should have a fairly small value.
- The batch_size determines how many elements each batch should have. The data is often divided into batches so that training is faster and requires ...
