Measuring Complexity using Information

Abstract

Measuring complexity in multidimensional systems with high degrees of freedom and a variety of types of information, remains an important challenge. Complexity of a system is related to the number and variety of components, the number and type of interactions among them, the degree of redundancy, and the degrees of freedom of the system. Examples show that different disciplines of science converge in complexity measures for low and high dimensional problems. For low dimensional systems, such as coded strings of symbols (text, computer code, DNA, RNA, proteins, music), Shannon’s Information Entropy (expected amount of _information_ in an event drawn from a given distribution) and Kolmogorov‘s Algorithmic Complexity (the length of the shortest algorithm that produces the object as output), are used for quantitative measurements of complexity. For systems with more dimensions (ecosystems, brains, social groupings), network science provides better tools for that purpose. For complex highly multidimensional systems, none of the former methods are useful. Useful Information Φ (Information that produces thermodynamic free energy) can be related to complexity. Φ can be quantified by measuring the thermodynamic Free Energy F and/or useful Work it produces. Here I propose to measure Complexity as Total Information I, defined as the information of the system, including Φ, useless information or Noise N, and Redundant Information R. Measuring one or more of these variables allows quantifying and classifying complexity. Complexity and Information are two windows overlooking the same fundamental phenomenon broadening out tools to quantify both.