Hands-On Machine Learning for Algorithmic Trading

Measuring the similarity of documents

The representation of documents as word vectors assigns to each document a location in the vector space created by the vocabulary. Interpreting vector entries as Cartesian coordinates in this space, we can use the angle between two vectors to measure their similarity because vectors that point in the same direction contain the same terms with the same frequency weights.

The preceding diagram (the one on the right) illustrates—simplified in two dimensions—the calculation of the distance between a document represented by a vector d₁ and a query vector (either a set of search terms or another document) represented by the vector q.

Cosine similarity equals the cosine of the angle between the two vectors. ...