Unicode Explained

in Finnish, would be marked up as <span lang="en">Smith</span>in inside a document

in Finnish. This would be awkward to do even with good authoring tools, and it could

in practice make things worse. A speech synthesizer, for example, might pause between

the base word and the suffix, when it switches mode.

There are many other problems in using detailed language markup. Thus, it is best to

limit it to major parts of a document only, such as expressions longer than a few words.

Language Codes

In order to express the language of some text in a machine-processable way, we need

a system of language codes. Preferably, the codes should be easy to recognize in a

program, but most importantly, they need to be systematic. We cannot really work

with information about language expressed in everyone’s own style and language, like

“English,” “anglais,” or “engl.”

The confusion of codes

Just as there is a confusion of languages in the world, there is a confusion of language

codes. Several incompatible systems are used to encode information about language in

a short identifier, typically a two- or three-letter alphabetic code or a number. To some

extent, the codes can be mapped to each other. However, there is no universally ac-

cepted list of languages, or anything close to that. Language code systems in use include:

• The ISO 639 standard (see below), with two- and three-letter alphabetic codes as

well numeric ...