Lossless Compression Handbook

Contents

List of Contributors

Preface

Part I:

Theory

Chapter 1" Information Theory behind Source Coding

Frans M. J. Willems and Tjailing J. Tjalkens

1.1

1.2

1.3

1.4

Introduction

1.1.1 Definition of Entropy

1.1.2 Properties of Entropy

1.1.3 Entropy as an Information Measure

1.1.4 Joint Entropy and Conditional Entropy

1.1.5 Properties of Joint Entropy and Conditional Entropy

1.1.6 Interpretation of Conditional Entropy

Sequences and Information Sources

1.2.1 Sequences

1.2.2 Information Sources

1.2.3 Memoryless Sources

1.2.4 Binary Sources

1.2.5 Discrete Stationary Sources

1.2.6 The Entropy Rate

Variable-Length Codes for Memoryless ...