Python Deep Learning - Second Edition

Word-based models

A word-based language model defines a probability distribution over sequences of words. Given a sequence of words of length m, it assigns a probability P(w₁, ... , w_m) to the full sequence of words. We can use these probabilities as follows:

• To estimate the likelihood of different phrases in natural language processing applications.
• As a generative model to create new text. A word-based language model can compute the likelihood of a given word to follow a sequence of words.