CJKV Information Processing, 2nd Edition

600

|

Chapter 9: Information Processing Techniques

Character insertion example—Shi-JISTable 9-13.

Text representation Shift-JIS representation

Original string



   

89  96  8  8 9

Correct



    

89  96  82 6 8  8 9

Incorrect



     

89  96 82 C6  8  8 9

Notice how the two-byte character  is split right down the middle, and that unexpect-

ed characters have resulted, two of which are interpreted as half-width katakana. Now

you can see why incorrect character insertion must never be allowed to happen—it leads

to corruption and data loss. Integrity is retained only with proper handling of two-byte

characters.

Table 9-14 provides this same example, but this time EUC-JP–encoded. Notice how dif-

ferent characters result from incorrect insertion—the expected  string becomes the

unexpected .

Character insertion example—EUC-JPTable 9-14.

Text representation EUC-JP representation

Original string



   

2    4 1  

Correct



    

2    4 8 4 1  

Incorrect



    

2   A4 C8  4 1  

e solution to this problem is simply to have the cursor move one or more bytes—the

number of bytes to move corresponds to the number of bytes used to represent the cur-

rent character.

Character Searching

e various instances of the grep program represent the most commonly used utilities

on Unix and some other platforms—grep ...