Paradoxical replay can protect contextual task representations from destructive interference when experience is unbalanced

Abstract

SummaryExperience replay is a powerful mechanism to learn efficiently from limited experience. Despite several decades of compelling experimental results, the factors that determine which experiences are selected for replay remain unclear. A particular challenge for current theories is that on tasks that feature unbalanced experience, rats paradoxically replay the less-experienced trajectory. To understand why, we simulated a feedforward neural network with two regimes: rich learning (structured representations tailored to task demands) and lazy learning (unstructured, task-agnostic representations). Rich, but not lazy, representations degraded following unbalanced experience, an effect that could be reversed with paradoxical replay. To test if this computational principle can account for the experimental data, we examined the relationship between paradoxical replay and learned task representations in the rat hippocampus. Strikingly, we found a strong association between the richness of learned task representations and the paradoxicality of replay. Taken together, these results suggest that paradoxical replay specifically serves to protect rich representations from the destructive effects of unbalanced experience, and more generally demonstrate a novel interaction between the nature of task representations and the function of replay in artificial and biological systems.HighlightsWe provide an explicit normative explanation and simulations of the experimentally observed puzzle of “paradoxical replay”, which we show can serve to protect certain task representations from the destructive effects of unbalanced experienceWe confirm with new analyses the main prediction of the theory, that “rich” task representations, measured using representational distance in the rodent hippocampus, show more paradoxical replay compared to “lazy” task representationsOur theory refines the notion of consolidation in complementary learning systems theory in showing that not all task representations benefit equally from interleaving, and provides an example of how the use of replay in artificial neural networks can be optimized