Information gain as uncertainty reduction

Abstract

It is shown that the various information content or information gain measures can be expressed in terms of the difference between the a priori and a posteriori uncertainties. Measures based on uncertainty expressed by means of variance are compared with those where the uncertainty also involves bias of the results, and it is shown that measures of the two kinds somewhat different application.