The fractional Kullback–Leibler divergence

Abstract

Abstract The Kullback–Leibler divergence or relative entropy is generalised by deriving its fractional form. The conventional Kullback–Leibler divergence as well as other formulations emerge as special cases. It is shown that the fractional divergence encapsulates different relative entropy states via the manipulation of the fractional order and for this reason it is the evolution equation for relative entropy. The fractional Kullback–Leibler divergence establishes mathematical dualities with other divergences or distance metrics. The fractional-order can be characterised as a distance metric between divergences or relative entropy states. Generalised asymptotic divergences and densities are derived that are mixtures of known approaches.