Skip to Main Content
Unicode Explained
book

Unicode Explained

by Jukka K. Korpela
June 2006
Beginner content levelBeginner
688 pages
26h 18m
English
O'Reilly Media, Inc.
Content preview from Unicode Explained
If a code point in a surrogate range is encountered in processing Unicode data (assuming
it has been decoded from an eventual encoding such as UTF-16), the situation should
be handled as an error. If it’s not a high surrogate immediately followed by a low sur-
rogate, there might be no way to handle the situation meaningfully, since we cannot
know what happened. But if there is a surrogate pair, odds are that the data was in fact
UTF-16 encoded and it was not decoded properly, so you might interpret the data
according to UTF-16.
When using UTF-8 (8-bit code units) or UTF-32 (32-bit code units), there is no use for
surrogates in any sense.
Unassigned Code Points and Private Use
Unassigned code points are simply points that have neither been allocated for any use
nor declared as noncharacters or private use points. You might visualize them as white
areas on a map, or as unoccupied rooms in the coding space. Programmers often use
such “free” positions for their own purposes, but that would be wrong here; the unas-
signed code points are not free at all. They are reserved for eventual future extensions.
By using unassigned code points, you would violate the Unicode standard. On the
practical side, you would take an unnecessary risk. It is quite possible that a future
version of Unicode will assign a specific meaning to the code point. This would involve
properties that you cannot anticipate.
Even ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Unicode Demystified

Unicode Demystified

Richard Gillam
Fonts & Encodings

Fonts & Encodings

Yannis Haralambous
Core Java™ Data Objects

Core Java™ Data Objects

Sameer Tyagi, Keiron McCammon, Michael Vorburger, Heiko Bobzin

Publisher Resources

ISBN: 059610121XCatalog PageErrata