The Definitive ANTLR 4 Reference, 2nd Edition

Context-Sensitive Lexical Problems

Consider the phrase “Brown leaves in the fall.” It’s ambiguous because there are two interpretations. If we’re talking about trees, the phrase refers to nature’s photosynthesis engine. If, on the other hand, we’re discussing a certain Ms. Jane Brown, the context totally changes the function of those words. “Leaves” shifts from a noun to a verb.

This situation resembles the problems we solved in ​Recognizing Ambiguous Phrases​ where context-sensitive C++ phrases like T(0) could be function calls or type casts depending on how T was defined elsewhere in the program. Such syntactic ambiguities arose because our C++ lexer sent vague generic ID tokens to the parser. We needed semantic predicates ...