Local prediction-learning enables neural networks to plan

Abstract

AbstractThe capability to plan a sequence of actions toward a given goal is a cornerstone of higher cognitive function. But compelling models for planning in the brain, or more generally in any type of neural network, are missing. We present a simple model for planning in neural networks, the Cognitive Map Learner (CML), that can achieve high performance on a variety of tasks by learning a cognitive map of the environment. The way how the CML constructs a cognitive map is based on a fundamental insight from neuroscience: Observations from the environment acquire meaning for the organism primarily through performing actions that change them. The CML also provides a viable alternative to reinforcement learning in robotics, since it learns faster and becomes more flexible, due to its task-agnostic design principles. The design of the CML is also of interest from the perspective of the relationship between self-attention networks (Transformers) and neural networks, since it combines attractive features of both.