Computation or Weight Adaptation? Rethinking the Role of Plasticity in Learning

Abstract

AbstractThe human brain is an adaptive learning system that can generalize to new tasks and unfamiliar environments. The traditional view is that such adaptive behavior requires a structural change of the learning system (e.g., via neural plasticity). In this work, we use artificial neural networks, specifically large language models (LLMs), to challenge the traditional view about the role of plasticity in learning and suggest that such an adaptive behavior can be achieved solely through computation if the learning system is suffciently trained. We focus on statistical learning paradigms. These require identifying underlying regularities in seemingly arbitrary word sequences and are largely considered to require neural plasticity. LLMs can capture arbitrary structures without weight adaptation despite the divergence from their natural language training data. Our work provides novel insights into the role of plasticity in learning, showing that suffciently trained learning systems are highly flexible, adapting to new tasks and environments solely through computation, much more than previously acknowledged. Furthermore, our work opens the door for future research to use deep learning models to conjure hypotheses about the brain.