
CHAPTER 3
Character Sets and Encodings
The world runs on a wide variety of character sets. This chapter describes the many
encodings for these sets and lists the characters in them. We also describe how con-
versions between the encodings can be performed, either with the functions of com-
monly used programs or separate converters. This chapter also discusses practical use
of the character sets in different contexts, such as email, Internet discussion forums,
and document interchange.
The use of Unicode does not mean that you need not know anything about encodings.
You will inevitably encounter non-Unicode data as well, and you need to work with it,
even if this only means converting it into Unicode. Moreover, Unicode itself can be
represented in different encodings, such as UTF-8 and UTF-16.
Mostly you don't neeed to know about the details of encodings. You certainly don't
have to know the code numbers of characters in each encoding, let alone memorize
them. What you need is an overview of the world of encodings, general information
about the suitability of each encoding for various purposes, and tools for mapping
between encodings.
The presentation of encodings in this chapter is practical rather than historical. For
history, one place to refer to is “A Brief History of Character Codes in North America,
Europe, and East Asia” at http://tronweb.super-nova.co.jp/characcodehist.html.
As explained ...