book

Advanced Perl Programming, 2nd Edition

by Simon Cozens

June 2005

Intermediate to advanced

298 pages

7h 28m

English

O'Reilly Media, Inc.

Read now

Unlock full access

AudienceContentsConventions Used in This BookUsing Code ExamplesWe’d Like to Hear from YouSafari® EnabledAcknowledgments
1.1. Introspection1.1.1. Preparatory Work: Fun with Globs1.1.1.1. Aliasing1.1.1.2. Accessing parts of a glob1.1.1.3. Creating subroutines with glob assignment1.1.2. AUTOLOAD1.1.3. CORE and CORE::GLOBAL1.1.4. Case Study: Hook::LexWrap1.1.5. Introspection with B1.2. Messing with the Class Model1.2.1. UNIVERSAL1.2.2. Dynamic Method Resolution1.2.3. Case Study: Singleton Methods1.3. Unexpected Code1.3.1. Overloading1.3.1.1. Simple operator overloading1.3.1.2. Other operator overloading1.3.1.3. Non-operator overloading1.3.2. Time Shifting1.3.2.1. Doing things now with eval/BEGIN1.3.2.2. Doing things later with CHECK1.3.2.3. Doing things at the end with DESTROY1.3.2.4. Case study: Acme::Dot1.4. Conclusion
2.1. Parse::RecDescent Grammars2.1.1. Simple Parsing with Parse::RecDescent2.1.1.1. Types of match2.1.1.2. Actions2.1.2. Debugging2.1.3. More Difficult Parsing2.1.3.1. Commit, reject, and error2.1.3.2. Syntactic whitespace2.1.3.3. Automating the process2.1.3.4. And much more...2.1.4. Some Examples2.1.4.1. Parsing iCalendar data2.1.4.2. .procmailrc2.2. Parse::Yapp2.3. Other Parsing Techniques2.3.1. HTML::Parser2.3.2. XML Parsing2.3.3. And Everything Else...2.4. Conclusion
3.1. Formats and Text::Autoformat3.1.1. Text::Autoformat3.2. Text::Template3.2.1. Loops, Arrays, and Hashes3.2.2. Security and Error Checking3.2.3. Text::Template Tricks3.3. HTML::Template3.3.1. Variables and Conditions3.3.2. Loops3.3.3. RSS Aggregation3.4. HTML::Mason3.4.1. Basic Components3.4.2. Basic Dynamism3.4.3. Perl Blocks3.4.4. Our RSS Aggregator3.5. Template Toolkit3.5.1. Filters3.5.2. Plugins3.5.3. Components and Macros3.5.4. The RSS Aggregator3.6. AxKit3.7. Conclusion
4.1. Beyond Flat Files4.2. Object Serialization4.2.1. Our Schema and Classes4.2.2. Dumping Data4.2.3. Storing and Retrieving Data4.2.4. The Pruning Problem4.2.5. Multilevel DBMs4.2.6. Pixie4.3. Object Databases4.3.1. Object Database Pitfalls4.3.2. Tangram4.3.2.1. Tangram CRUD: create, read, update, delete4.4. Database Abstraction4.4.1. Trivial Mapping4.4.2. Class::DBI4.4.2.1. Relationships4.4.2.2. Class::DBI extensions4.4.3. Other Contenders4.5. Practical Uses in Web Applications4.5.1. Class::DBI and the Template Toolkit4.5.2. Maypole4.5.3. Other Application Frameworks4.6. Conclusion
5.1. Perl and Natural Languages5.2. Handling English Text5.2.1. Pluralizations and Inflections5.2.2. Converting Words to Numbers5.3. Modules for Parsing English5.3.1. Splitting Up Text5.3.2. Stemming and Stopwording5.4. Categorization and Extraction5.4.1. Bayesian Analysis5.4.2. Keyword Extraction and Summary5.4.2.1. Keyword extraction5.4.3. Extracting Names and Places5.5. Conclusion
6.1. Terminology6.2. What Is Unicode?6.2.1. What Is UCS?6.2.2. What is the Unicode Consortium?6.2.3. Why Should I Care?6.3. Unicode Transformation Formats6.3.1. UCS-26.3.2. UTF-86.3.3. UTF-16BE6.3.4. UTF-16LE6.3.5. UCS-46.3.6. UTF-326.3.7. UTF-EBCDIC6.3.8. UTF-76.4. Handling UTF-8 Data6.4.1. Entering Unicode Characters6.4.2. Unicode Regular Expressions6.5. Encode6.5.1. The PerlIO Trick6.5.2. The Gory Details6.6. Unicode for XS Authors6.6.1. Traversing Strings6.6.2. Encoding Strings6.7. Conclusion
7.1. Programming in an Event-Driven Environment7.1.1. Hello, POE7.1.2. Hello, Again, POE!7.1.3. Wheels7.1.4. A Port Forwarder7.2. Top-Level Pieces: Components7.2.1. Medium-Level Components7.2.2. A POE Web Server7.2.3. Highest-Level Components7.3. Conclusion
8.1. Test::Simple8.2. Test::More8.2.1. Skips and Todos8.2.2. Automated Tests8.3. Test::Harness8.4. Test::Builder8.5. Test::Builder::Tester8.6. Keeping Tests and Code Together8.7. Unit Tests8.7.1. Test::Class8.7.2. Test::MockObject8.7.3. Testing Apache, DBI, and Other Complex Environments8.8. Conclusion
9.1. Simple Inline::C9.1.1. Taking and Giving9.1.2. C is not Always a Win9.2. More Complex Tasks with Inline::C9.2.1. Dealing with Perl’s Internal Values9.2.2. Handling the Stack9.2.3. Handling More Complex Perl Types9.2.3.1. References9.2.3.2. Arrays9.2.3.3. Hashes9.2.4. Wrapping C Libraries9.2.5. Debugging Inline Extensions9.2.6. Packaging Inline Modules9.3. Inline:: Everything Else9.3.1. Inline::Python9.3.2. Inline::Ruby9.3.3. Inline::CPR9.3.4. Inline::Struct9.3.5. Miscellaneous Other Inlines9.4. Conclusion

10.1. Obfuscation10.2. Just Another Perl Hacker10.3. Perl Golf10.4. Perl Poetry10.5. Acme::*10.6. Conclusion

Content preview from Advanced Perl Programming, 2nd Edition

Chapter 6. Perl and Unicode

Over the last couple of major releases, Perl has gained more advanced support for Unicode data manipulation. With the Perl 5.8 series, this support is now mature, so it’s worth taking some time to look at what Unicode means for your applications and what tools Perl hands you to deal with it.

Terminology

It’s a good idea to take a little time out, before we think about what Unicode is and what problem it solves, to clarify in our minds a few terms that have been widely used and abused in the programming world. In particular, the term character set is more troublesome than it might appear.

We often talk about the ASCII character set, but this relates to many different ideas—it could mean the actual suite of characters involved, or the order in which they are placed in that suite, or the way that a piece of text is represented in bytes. In fact, when people talk about text from an ASCII system, it may not even be ASCII. The potential for confusion comes because ASCII is a seven-bit character set, whereas for the past 25 years or so, computers have had eight-bit bytes. ASCII only defines the meaning of the first 128 entries in the set, so what should be done with the other 128? Rather than leave them unused and wasted, nearly every ASCII system chooses to define them in some way, usually with accented characters and extra symbols. Many manufacturers chose to make their machines use one of the range of national sets as defined by ISO standard 8859. Of these sets, ...