Embed Notice

Just a minor fact that doesn't seem to be stated in many places - maybe it's too obvious to state:

In the absence of knowledge of what a probability distribution is, people sometimes like to pick the maximum entropy distribution consistent with the knowledge they do have.

More generally you may have some prior idea of what the distribution is. In that case you might pick the distribution with the maximum entropy *relative* to your prior guess consistent with any new knowledge.

A special case is when you draw from a joint distribution on X and Y and observe X. The posterior distribution of Y is given by the conditional distribution P(Y|X). This is precisely the maximum entropy distribution relative to the original distribution with the constraint that Y is known.

So conditional probabilities are in fact maximum (relative) entropy distributions.