O'Reilly logo

The Definitive ANTLR 4 Reference, 2nd Edition by Terence Parr

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Broadcasting Tokens on Different Channels

Most programming languages ignore whitespace and comments in between tokens, which means they can appear anywhere. That presents a problem for a parser since it has to constantly check for optional whitespace and comment tokens. The common solution is to simply have the lexer match those tokens but throw them out, which is what we’ve done so far in this book. For example, our Cymbol grammar from Parsing Cymbol threw out whitespace and comments using the skip lexer command.

examples/Cymbol.g4
 
WS : [ \t\n\r]+ -> skip ;
 
 
SL_COMMENT
 
: ​'//'​ .*? ​'\n'​ -> skip
 
;

That works great for many applications, such as compilers, because the comments don’t affect code generation. If, on the other hand, we’re ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required