Unicode Explained

If a code point in a surrogate range is encountered in processing Unicode data (assuming

it has been decoded from an eventual encoding such as UTF-16), the situation should

be handled as an error. If it’s not a high surrogate immediately followed by a low sur-

rogate, there might be no way to handle the situation meaningfully, since we cannot

know what happened. But if there is a surrogate pair, odds are that the data was in fact

UTF-16 encoded and it was not decoded properly, so you might interpret the data

according to UTF-16.

When using UTF-8 (8-bit code units) or UTF-32 (32-bit code units), there is no use for

surrogates in any sense.

Unassigned Code Points and Private Use

Unassigned code points are simply points that have neither been allocated for any use

nor declared as noncharacters or private use points. You might visualize them as white

areas on a map, or as unoccupied rooms in the coding space. Programmers often use

such “free” positions for their own purposes, but that would be wrong here; the unas-

signed code points are not free at all. They are reserved for eventual future extensions.

By using unassigned code points, you would violate the Unicode standard. On the

practical side, you would take an unnecessary risk. It is quite possible that a future

version of Unicode will assign a specific meaning to the code point. This would involve

properties that you cannot anticipate.

Even ...