Affiliation:
1. Department of Mathematics, Imperial College London, South Kensington Campus , London SW7 2AZ, UK
Abstract
Summary
Direct use of the likelihood function typically produces severely biased estimates when the dimension of the parameter vector is large relative to the effective sample size. With linearly separable data generated from a logistic regression model, the loglikelihood function asymptotes and the maximum likelihood estimator does not exist. We show that an exact analysis for each regression coefficient produces half-infinite confidence sets for some parameters when the data are separable. Such conclusions are not vacuous, but an honest portrayal of the limitations of the data. Finite confidence sets are only achievable when additional, perhaps implicit, assumptions are made. Under a notional double-asymptotic regime in which the dimension of the logistic coefficient vector increases with the sample size, the present paper considers the implications of enforcing a natural constraint on the vector of logistic transformed probabilities. We derive a relationship between the logistic coefficients and a notional parameter obtained as a probability limit of an ordinary least-squares estimator. The latter exists even when the data are separable. Consistency is ascertained under weak conditions on the design matrix.
Publisher
Oxford University Press (OUP)
Subject
Applied Mathematics,Statistics, Probability and Uncertainty,General Agricultural and Biological Sciences,Agricultural and Biological Sciences (miscellaneous),General Mathematics,Statistics and Probability
Reference37 articles.
1. On the existence of maximum likelihood estimates in logistic regression models;Albert;Biometrika,1984
2. Methodologies in spectral analysis of large dimensional random matrices, a review;Bai;Statist. Sinica,1999
3. The information available in small samples;Bartlett;Proc. Camb. Phil. Soc,1936
4. Properties of sufficiency and statistical tests;Bartlett;Proc. R. Soc. Lond. A,1937
5. On the linear in probability model for binary data;Battey;R. Soc. Open Sci.,2019