Diagnosis of paired age agreement: a simulation of accuracy and precision effects

Abstract

Abstract In a typical age and growth study, multiple estimates of individual fish age are generated: by readers, methods, etc. These samples of paired data are evaluated with three complementary approaches: (i) tabulate or graph the data to illustrate patterns, (ii) calculate indices of precision to evaluate repeatability, and more recently, (iii) use tests of symmetry to evaluate bias. Herein, I simulated age data to evaluate the diagnostic power of these different approaches in terms of four types of accuracy: (i) no bias between known and estimated age, (ii) bias by adding 1 year to the estimated age, (iii) bias by subtracting 10% from the estimated age, and (iv) both biases together. Precision of estimated ages (standard deviation = 5, 10, and 15% of known age) was varied for each type of bias. Age-bias plots had difficulty detecting bias in ages classes with few (e.g. ≤5) individuals sampled. A Bland–Altman-bubble plot is introduced as an alternative to age-bias plots. In these simulations, the various indices of precision were either not diagnostic (i.e. per cent agreement) or redundant (average per cent error could be predicted from Chang's coefficient of variation). Evans and Hoenig's test and Bowker's test of symmetry performed well in most simulations, but each has specific weaknesses. Bowker's test was superior by having marginally fewer type I error rates; however, it had much worse type II error rates. McNemar's test offered no additional interpretative power in these simulations. All tests of symmetry had difficulty diagnosing bias when precision was low. These simulated results support continued use of all three approaches because they are complementary. Additional simulations would be helpful to confirm that these guidelines are relevant in other representative cases.