- It can be shown that
.**H_X(g(X,Y)) <= H_X(Y)**

a. Give a simple example where the inequality is strict.

b. Give conditions which guarantee equality.

- Find a really simple example where
.**H({(f(X),g(Y))}) < H(X x Y)** - Prove
by some means other than that used in class.**H_X(Y) + H_{X x Y}(Z) = H_X(Y x Z)** - Use the chain rule to show that if
,**m < n**and determine when equality occurs.**H(X_1,...,X_m) <= H(X_1,...,X_n)**

- Prove directly that
. Interpret.**H_X(Z) >= H_{X x Y}(Z)**

- Suppose that 75% of women are dark haired and 25% are blond.
Suppose also that 70% of men are dark haired and 30% are blond.
a. If all blond women marry dark haired men, how much information does a womans hair color give about her husband's?

b. Answer the same question if all blond women marry blond men.

c. Comment.

- (Ash, p. 25) Suppose that in a certain city, 3/4 of the male
high-school students
pass and 1/4 fail. Of those who pass, 10% own cars, while 50%
of the failing students own cars. All of the car owning students
belong to fraternities, while 40% of those who do not own cars but
pass, as well as 40% of those who do not own cars but fail, belong to
fraternities.

a. How much information is conveyed about a student's academic standing by specifying whether or not he owns a car?

b. How much information is conveyed about a student's academic standing by specifying whether or not he belongs to a fraternity?

c. How much information is conveyed about a student's academic standing by specifying both whether he owns a car and whether he belongs to a fraternity?

d. If a student is a member of a fraternity, how much information is conveyed about his academic standing by specifying whether or not he owns a car?

e. If a students academic standing, car-owning status, and fraternity status are transmitted by three successive bits, how much information is conveyed by each bit?

- Use the maximum entropy principle to find the probabilities for the
roll of a die in the following cases.

a. The roll of 6 is twice as likely as the roll of a 1.

b. Additionally, suppose that the average roll (after lots of trials) is 4.