Straight Line Fitting and Predictions: On a Marginal Likelihood Approach to Linear Regression and Errors-In-Variables Models

Abstract

Abstract Even in the simple case of univariate linear regression and prediction there are important choices to be made regarding the origins of the noise terms and regarding which of the two variables under consideration that should be treated as the independent variable. These decisions are often not easy to make but they may have a considerable impact on the results. A unified probabilistic (i.e., Bayesian with flat priors) treatment of univariate linear regression and prediction is given by taking, as starting point, the general errors-in-variables model. Other versions of linear regression can be obtained as limits of this model. The likelihood of the model parameters and predictands of the general errors-in-variables model is derived by marginalizing over the nuisance parameters. The resulting likelihood is relatively simple and easy to analyze and calculate. The well-known unidentifiability of the errors-in-variables model is manifested as the absence of a well-defined maximum in the likelihood. However, this does not mean that probabilistic inference cannot be made; the marginal likelihoods of model parameters and the predictands have, in general, well-defined maxima. A probabilistic version of classical calibration is also included and it is shown how it is related to the errors-in-variables model. The results are illustrated by an example from the coupling between the lower stratosphere and the troposphere in the Northern Hemisphere winter.