Jaynes Shannon's Constrained Ignorance and Surprise
share
In this simple article, with possible applications in theoretical and applied physics, we suggest an original way to derive the expression of Shannon's entropy from a purely variational approach,using constraints. Based on the work of Edwin T. Jaynes, our results are not fundamentally new but the context in which they are derived might, however, lead to a remarkably consistent formalism,where the maximum entropy principle appears naturally. After having given a general definition of "ignorance" in this framework, we derive the somehow general expected expression for the entropy using two approaches. In the first, one is biased and has a vague idea of the shape of the entropy function. In the second, we consider the general case, where nothing is a priori known. The merits of both ways of thinking are compared.
READ FULL TEXT