Reconciling Signal-Detection Models of Criterion Learning with the Generalized Matching Law

Abstract

AbstractTo make decisions that lead to favorable outcomes, animals have to consider both their perceptual uncertainty as well as uncertainty about the outcomes of their actions, such as reinforcements. There is a long tradition of research investigating how the reinforcement structure of a task controls animals’ response behavior. The relation between reinforcement and response rates has been described by the matching law and its generalizations for tasks with and without perceptual uncertainty. The influence of perceptual uncertainty on decision behavior is traditionally modeled with signal detection theory, which posits that a decision criterion is placed on an internal evidence axis. Where this criterion is placed and how it is updated based on reinforcements are open questions within signal detection theory. Various criterion learning models have been proposed; however, their steady-state behavior across different experimental conditions is not consistent with the aforementioned empirical matching laws. Here, we integrate models of criterion learning from signal detection theory with matching laws from animal learning theory to gain a better understanding of the mechanisms by which reinforcements and perceptual uncertainty jointly shape behavior. To do so, we first derive the criterion position that leads to behavior aligned with those laws. We then develop a model that updates the decision criterion trial by trial to learn this criterion position. Our model fits data from a previous experiment well and generates behavior in simulations that is in line with matching laws for perceptual tasks and the subjects’ behavior in the experiment.