Hands-On Machine Learning for Algorithmic Trading

Linguistic annotation

Linguistic annotations include the application of syntactic and grammatical rules to identify the boundary of a sentence despite ambiguous punctuation, and a token's role in a sentence for POS tagging and dependency parsing. It also permits the identification of common root forms for stemming and lemmatization to group related words:

• POS annotations: It helps disambiguate tokens based on their function (this may be necessary when a verb and noun have the same form), which increases the vocabulary but may result in better accuracy.
• Dependency parsing: It identifies hierarchical relationships among tokens, is commonly used for translation, and is important for interactive applications that require more advanced language ...