What Loss Functions Do Humans Optimize When They Perform Regression and Classification

Abstract

AbstractStudying how humans perceive patterns in visually presented data is useful for understanding data-based decision-making and potentially understanding visually mediated sensorimotor control. We conducted experiments to examine how human subjects perform the simplest machine learning or statistical estimation tasks: linear regression and binary classification on 2D scatter plots. We used inverse optimization to infer the loss function humans optimize when they perform these tasks. Minimizing the sum of regression error raised to the power of 1.7 best-described human performing regression on sparse data. Loss functions with lower exponents, which are less sensitive to outliers, were better descriptors for regression tasks performed on less sparse data consisting of more data points. For the classification task, minimizing a logistic loss function was on average a better descriptor of human choices than an exponential loss function applied to only misclassified data. People changed their strategies as data density increased. These results represent overall trends across subjects and trials but there was large inter- and intra-subject variability in human choices. Future work may examine other loss function families and other tasks. Such understanding of human loss functions may inform the design of applications that interact with humans better and imitate humans more effectively.