Shortcomings of Transfer Entropy and Partial Transfer Entropy: Extending Them to Escape the Curse of Dimensionality

Abstract

Transfer entropy (TE) captures the directed relationships between two variables. Partial transfer entropy (PTE) accounts for the presence of all confounding variables of a multivariate system and infers only about direct causality. However, the computation of partial transfer entropy involves high dimensional distributions and thus may not be robust in case of many variables. In this work, different variants of the partial transfer entropy are introduced, by building a reduced number of confounding variables based on different scenarios in terms of their interrelationships with the driving or response variable. Connectivity-based PTE variants utilizing the random forests (RF) methodology are evaluated on synthetic time series. The empirical findings indicate the superiority of the suggested variants over transfer entropy and partial transfer entropy, especially in the case of high dimensional systems. The above findings are further highlighted when applying the causality measures on financial time series.