Deep Learning for Computer Vision

Quantizing the models

The weights of deep learning models have 32-bit float values. When the weights are quantized to 8-bit, the decrease in accuracy is small and hence cannot be noticed in deployment. The precision of weights on the results seems to have fewer effects on accuracy performance of deep learning models. This idea is interesting about deep learning and useful when the model size becomes critical. By replacing 32-bit float values with 8-bit values, the model size can be significantly decreased and the inference speed can be increased. There are plenty of options when implementing the quantization of models. The weights can be stored in 8-bit but inference operations can be performed in 32-bit float values. Every component of the ...