Information Theory

Fifth Assignment: due Fri. Oct. 19

1. PI 6.6
2. PI 6.7
3. PI 6.8
4. PI 6.9
5. PI 6.14
6. PI 6.16
   
   
7. Consider the following stacked deck of cards. Take a normal deck and add an extra royal flush of spades, and 3 royal flushes of hearts. There are 72 cards in all.

   a. What is the entropy of the resulting deck?

   b. Verify the grouping law by taking the partition of the deck by rank.
   
   

8. Let's flip a fair coin, but after each flip the probability that the coin shows the same face on the next flip is 2/3.

   a. What is the joint probability on two flips? What is the reverse channel?

   b. Compute the entropy of two flips.

   c. Now consider three flips: What is the joint distribution? What is the channel from the first two flips to the third?

   d. Compute the entropy of three flips.

   e. Use an induction argument to compute the entropy of n flips.