3.5. Input Elements and Tokens
The input characters and line terminators that result from escape processing (§3.3) and then input line recognition (§3.4) are reduced to a sequence of input elements.
Input: InputElementsopt SuboptInputElements: InputElement InputElements InputElementInputElement: WhiteSpace Comment TokenToken: Identifier Keyword Literal Separator OperatorSub: the ASCII SUB character, also known as “control-Z”
Those input elements that are not white space (§3.6) or comments (§3.7) are tokens. The tokens are the terminal symbols of the syntactic grammar (§2.3).
White space (§3.6) and comments (§3.7) can serve to separate tokens that, if adjacent, might be tokenized in another manner. For example, ...