Dopamine regulates decision thresholds in human reinforcement learning

Abstract

AbstractDopamine fundamentally contributes to reinforcement learning by encoding prediction errors, deviations of an outcome from expectation. Prediction error coding in dopaminergic regions in human functional neuroimaging studies is well replicated. In contrast, replications of behavioral and neural effects of pharmacological modulations of the dopamine system in human reinforcement learning are scarce. Additionally, dopamine contributes to action selection, but direct evidence and process-specific accounts in human reinforcement learning are lacking. Here we examined dopaminergic mechanisms underlying human reinforcement learning in a within-subjects pharmacological approach in male human volunteers (n=31, within-subjects design; Placebo, 150mg L-dopa, 2mg Haloperidol) in combination with functional magnetic resonance imaging and a stationary reinforcement learning task. We had two aims. First, we aimed to replicate previously reported beneficial effects of L-dopa vs. Haloperidol on reinforcement learning from gains. This replication was not successful. We observed no performance benefit of L-Dopa vs. Haloperidol, and no evidence for alterations in neural prediction error signaling. In contrast, Bayesian analyses provided moderate evidence in favor of the null hypothesis. This unsuccessful replication is likely at least partly due to a number of differences in experimental design. Second, using combined reinforcement learning drift diffusion models, we tested the recent proposal that dopamine contributes to action selection by regulating decision thresholds. Model comparison revealed that the data were best accounted for by a reinforcement learning drift diffusion model with separate learning rates for positive and negative prediction errors. The model accounted for both reductions in RTs and increases in accuracy over the course of learning. The only parameter showing robust drug effects was the boundary separation parameter, which revealed reduced decision thresholds under both L-Dopa and Haloperidol, compared to Placebo, and the degree of threshold reduction accounted for individual differences in RTs between conditions. Results are in line with the idea that striatal dopamine regulates decision thresholds during action selection, and that lower dosages of D2 receptor antagonists increase striatal DA release via an inhibition of autoreceptor-mediated feedback mechanisms.