Introduction to Cryptography with Coding Theory, 3rd Edition

20.6 Exercises

1. Let $X_1$ and $X_2$ be two independent tosses of a fair coin. Find the entropy $H(X_1)$ and the joint entropy $H(X_1\text{,}{X}_2)$. Why is $H(X_1\text{,}{X}_2)=H(X_1)+H(X_2)$?

2. Consider an unfair coin where the two outcomes, heads and tails, have probabilities $p(heads)=p$ and $p(tails)=1-p$.

   1. If the coin is flipped two times, what are the possible outcomes along with their respective probabilities?

   2. Show that the entropy in part (a) is $-2p{log}_2(p)-2(1-p){log}_2(1-p)$. How could this have been predicted without calculating the probabilities in part (a)?

3. A random variable $X$ takes the values $1\text{,}2\text{,}\cdots \text{,}n\text{,}\cdots$ with probabilities $\frac{1}{2}\text{,}\frac{1}{2^2}\text{,}\cdots \text{,}\frac{1}{2^n}\text{,}\cdots$. Calculate the entropy $H(X)$.

4. Let $X$ be a random variable taking on integer values. The probability is 1/2 that $X$ is in the range