The Java® Language Specification, Java SE 7 Edition, Fourth Edition
by James Gosling, Bill Joy, Guy Steele, Gilad Bracha, Alex Buckley
3.5. Input Elements and Tokens
The input characters and line terminators that result from escape processing (§3.3) and then input line recognition (§3.4) are reduced to a sequence of input elements.
Input: InputElementsopt SuboptInputElements: InputElement InputElements InputElementInputElement: WhiteSpace Comment TokenToken: Identifier Keyword Literal Separator OperatorSub: the ASCII SUB character, also known as “control-Z”
Those input elements that are not white space (§3.6) or comments (§3.7) are tokens. The tokens are the terminal symbols of the syntactic grammar (§2.3).
White space (§3.6) and comments (§3.7) can serve to separate tokens that, if adjacent, might be tokenized in another manner. For example, ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access