Information Theory: Some Concepts and Measures

Abstract

The concepts of entropy and of information are increasingly used in spatial analysis. This paper analyses these ideas in order to show how measures of spatial distributions may be constructed from them. First, the information content of messages is examined and related to the notion of uncertainty. Then three information measures, due to Shannon, Brillouin, and Good, are derived and shown to be appropriate in analysing different spatial problems; in particular, the Shannon and Brillouin measures are extensively compared and the effects of sample size on them are investigated. The paper also develops appropriate multivariate analogues of the information measures. Finally, some comments are made on the relations between the concepts of entropy, information, and order.