Singularity Behaviour of the Density, Information, and Entropy Functions Defining a Uniform Non-stationary Stochastic Process

Abstract

Precise definitions and derivatives of the time-dependent continuous and discrete uniform probability density functions and related information and entropy functions are investigated. A stochastic system is formed that can represent a uniform noise source having a time-dependent variance and forming a uniform non-stationary stochastic process. The information and entropy function of the system are defined, and their properties are investigated in the time domain, including the limit cases defined for infinite and zero values of the time-dependent variance. In particular, the singularity properties of the entropy function will be investigated when the time-dependent variance reaches infinity. Like in thermodynamics, where the physical entropy of a system increases all the time, the information entropy of the stochastic system in information theory is also expected to increase towards infinity when the variance increases. All investigations are conducted for both the continuous and discrete random variables and their density functions. The presented theory is of particular interest in analyzing the Gaussian density function having infinite variance and tending to a uniform density function.