Deep Learning Essentials

Data normalization

Another good practice is to normalize real-valued input data. We do it by subtracting the mean and dividing the standard deviation:

>> X -= np.mean(X, axis = 0) # zero-center>>> X /= np.std(X, axis = 0) # normalize

Here, X is the input data. The reason why it helps is that each dimension of the input data may have a different range, and we use the same learning rate across all dimensions. Normalizing the data avoids the problem of over-compensating or under-compensating some dimensions.