There's more...
The idea behind the threshold (the encoding threshold, in our context) is that the parameter updates will happen across clusters, but only for the values that come under the user-defined limit (threshold). This threshold value is what we refer to as the encoding threshold. Parameter updates refer to the changes in gradient values during the training process. High/low encoding threshold values are not good for optimal results. So, it is reasonable to come up with a range of acceptable values for the encoding threshold. This is also termed as the sparsity ratio, in which the parameter updates happen across clusters.
In this recipe, we also discussed how to configure threshold algorithms for distributed training. The default ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access