Probability & Entropy

The value of information is ........... to the probability of its occurrence

• directly proportional

• inversely proportional

• has no relation

The range of any probability is from .....

• - 1 to 1

• • - 1 to 0

• 0 to 1

• 0 to - 1

To calculate the Entropy, it is the summation of the product between ...

• [1/P(x)] * I(x)

• P(x) * I(x)

In general, what is the unit of measurement for any Entropy?

Answer explanation

or r-ary bits

The mutual information is equal to ...

• H(X) - H(X|Y)

• H(Y) - H(X|Y)

• H(Y) - H(Y|X)

• H(X) - H(Y|X)

Can any mutual information be in negative value?

• Yes

• No

• Depending on the problem

You have to be familiar with it, to be able to answer the following

I (X, Y) ......... I (Y, X)

• Greater than

• Less than

• Equal

• No relation

H (X) ........ H (Y)

• Greater than

• Less than

• Equal

• No relation

H (Y) ........ H (X, Y)

• Greater than

• Less than

• Equal

• No relation