Java Deep Learning Cookbook

How it works...

In step 1, we configured the threshold algorithm in SharedTrainingMaster, where the default algorithm is AdaptiveThresholdAlgorithm. Threshold algorithms will determine the encoding threshold for distributed training, which is a hyperparameter that's specific to distributed training. Also, note that we are not discarding the rest of the parameter updates. As we mentioned earlier, we put them into separate residual vectors and process them later. We do this to reduce the network traffic/load during training. AdaptiveThresholdAlgorithm is preferred in most cases for better performance.

In step 2, we used ResidualPostProcessor to post process the residual vector. The residual vector was created internally by the gradient sharing ...