Machine Learning for Developers

Multiclass application – softmax regression

Up until now, we have been classifying in the case of only having two classes, or in probabilistic language, event occurrence probabilities p. But this logistic regression can also be conveniently generalized to account for many classes.

As we saw before, in logistic regression we assumed that the labels were binary (y(i)∈{0,1}), but softmax regression allows us to handle y(i)∈{1,…,K}, where K is the number of classes and the label y can take on K different values, rather than only two.

Given a test input x, we want to estimate the probability that P(y=k|x) for each value of k=1,…,K. The softmax regression will make this output a K-dimensional vector (whose elements sum to 1), giving us our K estimated ...