Understanding Compression

Foreword

When I first began programming, I had no idea what data compression was nor why it mattered. Luckily, my Apple II Plus computer came with 0.000048 GB of memory (48 KB), which was quite a lot in 1979, and was enough to let me explore programming and computer graphics without realizing that my programs and data were constantly being compressed and decompressed behind the scenes in order to reduce their size in memory. Thanks, Woz!

After programming for a few years, I had discovered:

• Data compression took time and could slow down my software.
• Changing my data organization could make the compressed data smaller.
• There are a bewildering variety of complicated data compression algorithms.

This led to the realization that compression was not a rigid black box; rather, it’s a flexible tool that greatly influenced the quality of my software and could be manipulated in several ways:

• Changing compression algorithms could make my software run faster.
• Pairing my data organization with the right compression algorithm could make my data smaller.
• Choosing the wrong data organization or algorithm could make my data larger (and/or run slower).

Ah! Now I knew why data compression mattered. If things weren’t fitting into memory or were decompressing too slowly, I could slightly change my data organization to better fit the compression algorithm. I’d simply put numbers together in one group, strings in another, build tables of recurring data types, or truncate fractions into integers. ...