Entropy of Artificial Intelligence

Abstract

We describe a model of artificial intelligence systems based on the dimension of the probability space of the input set available for recognition. In this scenario, we can understand a subset, which means that we can decide whether an object is an element of a given subset or not in an efficient way. In the machine learning (ML) process we define appropriate features, in this way shrinking the defining bit-length of classified sets during the learning process. This can also be described in the language of entropy: while natural processes tend to increase the disorder, that is, increase the entropy, learning creates order, and we expect that it decreases a properly defined entropy.