O'Reilly logo

DocBook 5: The Definitive Guide by Richard L. Hamilton, Norman Walsh

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Name

token — A unit of information

Synopsis

token ::= (text | phrase db._phrase | replaceable | Graphic inlines | Indexing inlines | Linking inlines | Ubiquitous inlines)*

Description

A token identifies a unit of information. Usually, tokens are the result of some processing pass that has performed lexical analysis and divided a data set into the smallest units of information used for subsequent processing.

Exactly what constitutes a token varies by context.

Processing expectations

Formatted inline.

See Also

classname, property, symbol, type

Examples

<article xmlns='http://docbook.org/ns/docbook'>
<title>Example token</title>

<para>In parsing, line ends are turned into the <token>CRLF</token>, all other
whitespace becomes <token>WHITESP</token>.
</para>

</article>

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required