
186 ◾ Computing
where K
B
is the Boltzmann constant, and each p
i
is the probability of a
microstate. is formula was given by Josiah Willard Gibbs (1839–1903)
in 1878 and is called Gibbs entropy.
21.3 SHANNON–FANO CODING AND HUFFMAN CODING
An ecient coding technique of messages was proposed in Shannon’s paper
in 1948. Robert Mario Fano (1917–) developed Shannon’s method and pub-
lished it as a technical report [11]. It is a technique for constructing a code
based on a set of symbols and their estimated probabilities, and was named
Shannon–Fano coding. Here, the set of symbols is divided into two sets whose
total probabilities are as close ...