Hands-On Machine Learning for Algorithmic Trading

Challenges of NLP

The conversion of unstructured text into a machine-readable format requires careful preprocessing to preserve valuable semantic aspects of the data. How humans derive meaning from, and comprehend the content of language, is not fully understood and improving language understanding by machines remains an area of very active research.

NLP is challenging because the effective use of text data for ML requires an understanding of the inner workings of language as well as knowledge about the world to which it refers. Key challenges include the following:

• Ambiguity due to polysemy; that is, a word or phrase can have different meanings depending on context (local high-school dropouts cut in half could be taken a couple of ways, ...