Hands-On Artificial Intelligence for Beginners

Training and optimizing VAEs

VAEs utilize a negative log-likelihood loss as their reconstruction loss to measure how much information is lost during the reconstruction phase of the decoder. If the decoder does not reconstruct the input satisfactorily, it will incur a large reconstruction loss. VAEs also introduce something called Kullback–Leibler (KL) divergence into their loss functions. KL divergence simply measures how much two probability distributions diverge; in other words, how different they are from one another. We want to minimize the KL distance between the mean and standard deviation of the target distribution and that of a standard normal. It is properly minimized when the mean is zero and the standard deviation is one. The log-likelihood ...