Algorithmic thermodynamics

Abstract

Algorithmic entropy can be viewed as a special case of the entropy studied in statistical mechanics. This viewpoint allows us to apply many techniques developed for use in thermodynamics to the subject of algorithmic information theory. In particular, suppose we fix a universal prefix-free Turing machine and letXbe the set of programs that halt for this machine. Then we can regardXas a set of ‘microstates’, and treat any function onXas an ‘observable’. For any collection of observables, we can study the Gibbs ensemble that maximises entropy subject to constraints on the expected values of these observables. We illustrate this by taking the log runtime, length and output of a program as observables analogous to the energyE, volumeVand number of moleculesNin a container of gas. The conjugate variables of these observables allow us to define quantities we call the ‘algorithmic temperature’T, ‘algorithmic pressure’Pand ‘algorithmic potential’ μ, since they are analogous to the temperature, pressure and chemical potential. We derive an analogue of the fundamental thermodynamic relationdE=TdS−PdV+μdN, and use it to study thermodynamic cycles analogous to those for heat engines. We also investigate the values ofT,Pand μ for which the partition function converges. At some points on the boundary of this domain of convergence, the partition function becomes uncomputable – indeed, at these points the partition function itself has non-trivial algorithmic entropy.