Learning Python, 5th Edition

Chapter 37. Unicode and Byte Strings

So far, our exploration of strings in this book has been deliberately incomplete. Chapter 4’s types preview briefly introduced Python’s Unicode strings and files without giving many details, and the strings chapter in the core types part of this book (Chapter 7) deliberately limited its scope to the subset of string topics that most Python programmers need to know about.

This was by design: because many programmers, including most beginners, deal with simple forms of text like ASCII, they can happily work with Python’s basic str string type and its associated operations and don’t need to come to grips with more advanced string concepts. In fact, such programmers can often ignore the string changes in Python 3.X and continue to use strings as they may have in the past.

On the other hand, many other programmers deal with more specialized types of data: non-ASCII character sets, image file contents, and so on. For those programmers, and others who may someday join them, in this chapter we’re going to fill in the rest of the Python string story and look at some more advanced concepts in Python’s string model.

Specifically, we’ll explore the basics of Python’s support for Unicode text—rich character strings used in internationalized applications—as well as binary data—strings that represent absolute byte values. As we’ll see, the advanced string representation story has diverged in recent versions of Python:

• Python 3.X provides ...