Unicode Explained

CHAPTER 7

Characters and Languages

The chapter describes some IT-related requirements of different languages and writing

systems,

such as how to deal with right-to-left writing (a common source of confusion).

This includes transliteration, transcription, and simplifications. The interaction be-

tween encoding, language, and font settings is described. Moreover, language codes,

language metadata, and language markup are described, illustrated with XML exam-

ples.

Information about the language of text is more important when using Unicode than

with older character codes. The reason is that the unification principle of Unicode

(described in Chapter 4) removes many distinctions between language-dependent var-

iants of characters. For example, Unicode often uses the same code position for a Chi-

nese character and a historically and semantically related but different Japanese char-

acter. To express the difference, you would include information about language—e.g.,

by using markup.

Writing Systems and IT

In information technology, we often deal with text just as any data, with no regard to

its internal structure or meaning. When sending a plain text file, for example, we con-

sider at most issues like efficiency, encoding, and checking that the data arrives un-

changed. However, operations like page layout, searching, indexing, and word pro-

cessing need to be sensitive at least to some features and ...