Deep Learning Essentials

Multi-class multi-label classification

For multi-label multi-class classification problems, that is, when each sample can have many correct answers, the sigmoid function is often used at the output layer of the neural network models (without applying softmax). With the probabilities of each class being independent from the other class probabilities, one can use the threshold for each class probability, so one sample may get multiple labels. Cross-entropy loss is still the most commonly used loss function in this case but would have a slightly different formula as compared to multi-class classification scenarios. In TensorFlow, one may choose sigmoid_cross_entropy_with_logits (https://www.tensorflow.org/api_docs/python/tf/nn/sigmoid_cross_entropy_with_logits ...