Theoretical Aspects on Measures of Directed Information with Simulations

Abstract

Measures of directed information are obtained through classical measures of information by taking into account specific qualitative characteristics of each event. These measures are classified into two main categories, the entropic and the divergence measures. Many times in statistics we wish to emphasize not only on the quantitative characteristics but also on the qualitative ones. For example, in financial risk analysis it is common to take under consideration the existence of fat tails in the distribution of returns of an asset (especially the left tail) and in biostatistics to use robust statistical methods to trim extreme values. Motivated by these needs in this work we present, study and provide simulations for measures of directed information. These measures quantify the information with emphasis on specific parts (or events) of their probability distribution, without losing the whole information of the less significant parts and at the same time by concentrating on the information of the parts we care about the most.