Deep Learning Quick Reference

Visualizing a broken network

TensorBoard is a great troubleshooting tool. To demonstrate this, I'm going to copy our deep neural network and break it! Luckily, breaking a neural network is really easy. Trust me, I've done it enough unintentionally that I'm basically an expert at this point.

Imagine that you have just trained a new neural network and seen that the loss looked like this:

The loss function for this network is stuck, and it's way higher than our previous run. What went wrong?

Navigate to the HISTOGRAMS section of TensorBoard and visualize the first hidden layer. Let's compare the histogram of the weights for hidden layer 1 in ...