May 2010
Intermediate to advanced
552 pages
13h 37m
English
token — A unit of information
token ::=
(text | phrase db._phrase | replaceable
| Graphic inlines | Indexing inlines | Linking inlines | Ubiquitous inlines)*
A token identifies a unit of information.
Usually, “tokens” are the result of some processing pass
that has performed lexical analysis and divided a data set into the
smallest units of information used for subsequent processing.
Exactly what constitutes a token varies by context.
Formatted inline.
classname, property, symbol, type
<article xmlns='http://docbook.org/ns/docbook'> <title>Example token</title> <para>In parsing, line ends are turned into the <token>CRLF</token>, all other whitespace becomes <token>WHITESP</token>. </para> </article>
Read now
Unlock full access