Hands-On Machine Learning for Algorithmic Trading

Automatic phrase detection

We use gensim to detect phrases as previously introduced. The Phrases module scores the tokens and the Phraser class transforms the text data accordingly. The notebook shows how to repeat the process to create longer phrases:

sentences = LineSentence(f'ngrams_1.txt')phrases = Phrases(sentences=sentences,                  min_count=25,  # ignore terms with a lower count                  threshold=0.5,  # only phrases with higher score                  delimiter=b'_',  # how to join ngram tokens                  scoring='npmi')  # alternative: defaultgrams = Phraser(phrases)sentences = grams[sentences]

The most frequent bigrams include common_stock, united_states, cash_flows, real_estate, and interest_rates.