R2 should not be used to describe behavioral‐economic discounting and demand models

Abstract

AbstractLiterature concerning operant behavioral economics shows a strong preference for the coefficient of determination (R2) metric to (a) describe how well an applied model accounts for variance and (b) depict the quality of collected data. Yet R2 is incompatible with nonlinear modeling. In this report, we provide an updated discussion of the concerns with R2. We first review recent articles that have been published in the Journal of the Experimental Analysis of Behavior that employ nonlinear models, noting recent trends in goodness‐of‐fit reporting, including the continued reliance on R2. We then examine the tendency for these metrics to bias against linear‐like patterns via a positive correlation between goodness of fit and the primary outputs of behavioral‐economic modeling. Mathematically, R2 is systematically more stringent for lower values for discounting parameters (e.g., k) in discounting studies and lower values for the elasticity parameter (α) in demand analysis. The study results suggest there may be heterogeneity in how this bias emerges in data sets of varied composition and origin. There are limitations when using any goodness‐of‐fit measure to assess the systematic nature of data in behavioral‐economic studies, and to address those we recommend the use of algorithms that test fundamental expectations of the data.