Skip to Content
Learning TypeScript 2.x - Second Edition
book

Learning TypeScript 2.x - Second Edition

by Remo H. Jansen
April 2018
Beginner content levelBeginner
536 pages
13h 21m
English
Packt Publishing
Content preview from Learning TypeScript 2.x - Second Edition

Lexemes and tokens

A lexeme is a sequence of characters in the source program that matches the pattern for a token. We can say that a token has a pattern and a pattern can be matched by many lexemes, in some cases. As a result, in a programming language, there are an infinite number of potential lexemes and a limited number of tokens.

The easiest way to understand the difference between a lexeme and a token is to take a look at an example, such as the following code snippet:

while (y >= t) y = y - 3; 

The preceding code snippet will be parsed into the following lexemes and tokens:

Lexeme

Token

while

WhileKeyword

(

OpenParenToken

y

Identifier

>=

GreaterThanEqualsToken

t

Identifier

)

CloseParenToken ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Mastering TypeScript - Fourth Edition

Mastering TypeScript - Fourth Edition

Nathan Rozentals
Learning TypeScript

Learning TypeScript

Josh Goldberg
TypeScript for Beginners

TypeScript for Beginners

Bharath Thippireddy

Publisher Resources

ISBN: 9781788391474Supplemental Content