Reward prediction-errors weighted by cue salience produces addictive behaviors in simulations, with asymmetrical learning and steeper delay discounting

Abstract

AbstractDysfunction in learning and motivational systems are thought to contribute to addictive behaviours. Previous models have suggested that dopaminergic roles in learning and motivation could produce addictive behaviours through pharmacological manipulations that provide excess dopaminergic signalling towards these learning and motivational systems. Redish 2004 suggested a role based on dopaminergic signals of value prediction error, while Zhang et al. 2009 suggested a role based on dopaminergic signals of motivation. Both these models present significant limitations. They do not explain the reduced sensitivity to drug-related costs/negative consequences, the increased impulsivity generally found in people with a substance use disorder, craving behaviours, and non-pharmacological dependence, all of which are key hallmarks of addictive behaviours. Here, we propose a novel mathematical definition of salience, that combines aspects of dopamine’s role in both, learning and motivation, within the reinforcement learning framework. Using a single parameter regime, we simulated addictive behaviours that the Zhang et al. 2009 and Redish 2004 models also produce but we went further in simulating the downweighting of drug-related negative prediction-errors, steeper delay discounting of drug rewards, craving behaviours and aspects of behavioural/non-pharmacological addictions. The current salience model builds on our recently proposed conceptual theory that salience modulates internal representation updating and may contribute to addictive behaviours by producing misaligned internal representations (Kalhan et al., 2021). Critically, our current mathematical model of salience argues that the seemingly disparate learning and motivational aspects of dopaminergic functioning may interact through a salience mechanism that modulates internal representation updating.