Unicode Explained

CHAPTER 10

Characters in Internet Protocols

This chapter

describes how character encoding information is transmitted in Internet

protocols, including MIME and HTTP, and how content negotiation works on the

Web, mainly for the purposes of negotiating on character encoding and language. This

constitutes a basis for a presentation of some fundamentals of multilingual web au-

thoring at the technical level. Moreover, the use of characters in the protocols them-

selves, such as in Internet message headers and URLs, is described, with focus on the

partial shift from pure ASCII to Unicode. In particular, the technical basis of Interna-

tionalized Domain Names and Internationalized URLs is described.

A common situation in which people first encounter problems with character encoding

is when they start authoring web pages in new languages. If you have a web site in

English, you might never think about encodings, since you can work with default set-

tings. Then, if you want to add a page in Japanese or Arabic, you meet several problems

at a time:

• What authoring tools (software) should I use?

• What fonts do I use?

• Which encoding should I use?

• How do I give information about the encoding?

• What tags should I put in my documents to tell the language I’m using?

Many of the difficulties in such situations arise from the common confusion of fonts,

encodings, and languages. Other chapters of this book have ...