COMPARISON OF STATISTICAL SIGNIFICANCE CRITERIA

Abstract

We study and compare two classes of statistical criteria to assess the significance of exceptional words. Indeed, the Z-score-like criteria, or the normal approximation that is a strict equivalent, suffer from several drawbacks in terms of sensitivity and specificity. Thanks to the combinatorial structure of words, a computation of the exact P-value has been made possible by recent mathematical results. We study here the drawbacks of the Z-score, the choice of the threshold and the tightness to the P-value. A major conclusion is that the normal approximation is always very poor and overestimates statistical significance.