Abstract
Models can be simple for different reasons: because they yield a simple and computationally efficient interpretation of a generic dataset (e.g., in terms of pairwise dependencies)—as in statistical learning—or because they capture the laws of a specific phenomenon—as e.g., in physics—leading to non-trivial falsifiable predictions. In information theory, the simplicity of a model is quantified by the stochastic complexity, which measures the number of bits needed to encode its parameters. In order to understand how simple models look like, we study the stochastic complexity of spin models with interactions of arbitrary order. We show that bijections within the space of possible interactions preserve the stochastic complexity, which allows to partition the space of all models into equivalence classes. We thus found that the simplicity of a model is not determined by the order of the interactions, but rather by their mutual arrangements. Models where statistical dependencies are localized on non-overlapping groups of few variables are simple, affording predictions on independencies that are easy to falsify. On the contrary, fully connected pairwise models, which are often used in statistical learning, appear to be highly complex, because of their extended set of interactions, and they are hard to falsify.
Subject
General Physics and Astronomy
Reference46 articles.
1. Big Data: A Revolution That Will Transform How We Live, Work and Think;Mayer-Schonberger,2013
2. The End of Theory: The Data Deluge Makes the Scientific Method Obsolete, 2008. Wired
https://www.wired.com/2008/06/pb-theory/
3. Are we there yet?
Cited by
4 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Simplicity science;Indian Journal of Physics;2024-02-01
2. Financial price dynamics and phase transitions in the stock markets;The European Physical Journal B;2023-03
3. Optimal work extraction and the minimum description length principle;Journal of Statistical Mechanics: Theory and Experiment;2020-09-04
4. Generic assembly patterns in complex ecological communities;Proceedings of the National Academy of Sciences;2018-02-13