Hands-On GPU Programming with Python and CUDA

Questions

1. Suppose you construct a DNN and after training it, it yields only garbage. After inspection, you find that all of the weights and biases are either huge numbers or NaNs. What might the problem be?
2. Name one possible problem with a small training_rate value.
3. Name one possible problem with a large training_rate value.
4. Suppose we want to train a DNN that will assign multiple labels to an image of an animal ("slimey", "furry", "red", "brown", and so on). Should we use a sigmoid or softmax layer at the end of the DNN?
5. Suppose we want to classify an image of a single animal as either a cat or dog. Do we use sigmoid or softmax?
6. If we decrease the batch size, will there be more or less updates to the weights and biases during gradient descent ...