Abstract
This paper shows that one cannot learn the probability of rare events without imposing further structural assumptions. The event of interest is that of obtaining an outcome outside the coverage of an i.i.d. sample from a discrete distribution. The probability of this event is referred to as the “missing mass”. The impossibility result can then be stated as: the missing mass is not distribution-free learnable in relative error. The proof is semi-constructive and relies on a coupling argument using a dithered geometric distribution. Via a reduction, this impossibility also extends to both discrete and continuous tail estimation. These results formalize the folklore that in order to predict rare events without restrictive modeling, one necessarily needs distributions with “heavy tails”.
Funder
National Science Foundation
Office of Naval Research
Subject
General Physics and Astronomy
Reference24 articles.
1. THE POPULATION FREQUENCIES OF SPECIES AND THE ESTIMATION OF POPULATION PARAMETERS
2. On the Convergence Rate of Good–Turing Estimators;McAllester,2000
3. Concentration inequalities for the missing mass and for histogram rule error;McAllester;J. Mach. Learn. Res.,2003
4. On the concentration of the missing mass
5. Rare Probability Estimation Under Regularly Varying Heavy Tails;Ohannessian;JMLR Proc.,2012
Cited by
11 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献