A LINK BETWEEN THE MAXIMUM ENTROPY APPROACH AND THE VARIATIONAL ENTROPY FORM

Abstract

The maximum entropy approach operating with quite general entropy measure and constraint is considered. It is demonstrated that for a conditional or parametrized probability distribution f(x|μ), there is a "universal" relation among the entropy rate and the functions appearing in the constraint. This relation allows one to translate the specificities of the observed behavior θ(μ) into the amount of information on the relevant random variable x at different values of the parameter μ. It is shown that the recently proposed variational formulation of the entropic functional can be obtained as a consequence of this relation, that is from the maximum entropy principle. This resolves certain puzzling points that appeared in the variational approach.