Embracing curiosity eliminates the exploration-exploitation dilemma

Abstract

Balancing exploration with exploitation is seen as a mathematically intractable dilemma that all animals face. In this paper, we provide an alternative view of this classic problem that does not depend on exploring to optimize for reward. We argue that the goal of exploration should be pure curiosity, or learning for learning’s sake. Through theory and simulations we prove that explore-exploit problems based on this can be solved by a simple rule that yields optimal solutions: when information is more valuable than rewards, be curious, otherwise seek rewards. We show that this rule performs well and robustly under naturalistic constraints. We suggest three criteria can be used to distinguish our approach from other theories.