Relationship Between Kendall's tau Correlation and Mutual Information

Abstract

Mutual information (MI) can be viewed as a measure of multivariate association in a random vector. However, the estimation of MI is difficult since the estimation of the joint probability density function (PDF) of non Gaussian distributed data is a hard problem. Copula function is an appropriate tool for estimating MI since the joint probability density function ofrandom variables can be expressed as the product of the associated copula density function and marginal PDF’s. With a little search, we find that the proposed copulas-based mutual information is much more accurate than conventional methods such as the joint histogram and Parzen window-based MI. In this paper, by using the copulas-based method, we compute MI forsome family of bivariate distribution functions and study the relationship between Kendall’s tau correlation and MI of bivariate distributions. Finally, using a real dataset, we illustrate the efficiency of this approach.