Recognizing Common Lexical Structures

Computer languages look remarkably similar lexically. For example, if I scramble up the order to obscure grammatical information, tokens ) 10 ( f could be combined into valid phrases from the earliest languages to the most recent. Fifty years ago, we’d see (f 10) in LISP and f(10) in Algol. Of course, f(10) is also valid in virtually all programming languages from Prolog to Java to the new Go language.[26] Lexically, then, functional, procedural, declarative, and object-oriented languages look pretty much the same. Amazing!

That’s great because we have to learn only how to describe identifiers and integers once and, with little variation, apply them to most programming languages. As with parsers, lexers ...

Get The Definitive ANTLR 4 Reference, 2nd Edition now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.