A New Method to Quantify the Causal Effects of Reinforcement in Terms of Behavioral Selection

Abstract

AbstractWe present a new methodology to partition different sources of learning within a selectionist framework based on the Price equation – the Multilevel Model of Behavioral Selection (MLBS). The MLBS provides a theoretical and formal background to disentangle behavior changes due to selection effects (“reinforcement”) from behavior changes due to non-selection effects (e.g., “effort” or “skill”). The model allows to quantify the causal effect of reinforcement even in the presence of opposing sources of behavior change and to test whether the reinforcement effect is statistically significant for a given subject.We demonstrate our method using training data from a reinforcement experiment with nine pigeons (Columba livia). The pigeons were trained to search for food in small holes on the ground of a special cage. In each session, the holes in only one half of the cage were filled with food (reinforced region), while the holes in the other half of the cage remained empty (nonreinforced region). Although the total time spent in the reinforced region between training sessions tends to decrease between training sessions, our method reveals positive behavioral selection (reinforcement effect) in most of the pigeons. However, this selection effect is masked by a strong non-selection component that consists in pigeons becoming more efficient foragers (skill effect) and in an increased peck frequency (effort effect).The partitioning of behavior change into selection and non-selection components allows to identify the causal effects of reinforcement even if they cannot be isolated experimentally. Our method is especially useful in the context of non-equilibrium behavior analysis (such as the change between training sessions), and in the context of behavior observations in naturalistic settings.