Quantifying and reducing researcher subjectivity in the generation of climate indices from documentary sources
-
Published:2022-05-17
Issue:5
Volume:18
Page:1071-1081
-
ISSN:1814-9332
-
Container-title:Climate of the Past
-
language:en
-
Short-container-title:Clim. Past
Author:
Adamson George C. D.ORCID, Nash David J.ORCID, Grab Stefan W.
Abstract
Abstract. The generation of index-based series of meteorological
phenomena, derived from narrative descriptions of weather and climate in
historical documentary sources, is a common method to reconstruct past
climatic variability and effectively extend the instrumental record. This
study is the first to explicitly examine the degree of inter-rater
variability in producing such series, a potential source of bias in
index-based analyses. Two teams of raters were asked to produce a
five-category annual rainfall index series for the same dataset, consisting
of transcribed narrative descriptions of meteorological variability for 11
“rain years” in nineteenth-century Lesotho, originally collected by Nash and Grab (2010). One group of raters (n = 71) was comprised of students studying
for postgraduate qualifications in climatology or a related discipline; the
second group (n = 6) consisted of professional meteorologists and
historical climatologists working in southern Africa. Inter-rater
reliability was high for both groups at r = 0.99 for the student raters and r = 0.94 for the professional raters, although ratings provided by the
student group disproportionately averaged to the central value (0:
normal/seasonal rains) where variability was high. Back calculation of
intraclass correlation using the Spearman–Brown prediction formula showed
that a target reliability of 0.9 (considered “excellent” in other published studies) could be obtained with as few as eight student raters and four professional raters. This number reduced to two when examining a subset of the professional group (n = 4) who had previously published historical climatology papers on southern Africa. We therefore conclude that variability between researchers should be considered minimal where
index-based climate reconstructions are generated by trained historical
climatologists working in groups of two or more.
Publisher
Copernicus GmbH
Subject
Paleontology,Stratigraphy,Global and Planetary Change
Reference64 articles.
1. Adamson, G. C. D. and Nash, D. J.: Documentary reconstruction of monsoon rainfall variability over western India, 1781–1860, Clim. Dynam., 42, 749–769, https://doi.org/10.1007/s00382-013-1825-6, 2014. 2. Alexandre, P.: Le climat en Europe au moyen âge: contribution à
l'histoire des variations climatiques de 1000 à 1425, d'après les
narratives de l'Europe Occidentale, Recherches d'histoire et de sciences
sociales 24, Éditions de l'École des hautes études en sciences
sociales, Paris, https://doi.org/10.1163/182539188x00438, 1987. 3. Baron, W. R.: Historical climate records from the northeastern United States,
1640 to 1900, in: Climate Since A.D. 1500, edited by: Bradley, R. S. and
Jones, P. D., Routledge, London, 74–91, ISBN 9781138178144, 1995. 4. Baron, W. R., Gordon, G. A., Borns, H. W., and Smith, D. C.: Frost-free record reconstruction for eastern Massachusetts, 1733–1980, J. Clim. Appl.
Meteorol., 23, 317–319, https://doi.org/10.1175/1520-0450(1984)023<0317:FFRRFE>2.0.CO;2, 1984. 5. Bauch, M., Labbé, T., Engel, A., and Seifert, P.: A prequel to the Dantean Anomaly: the precipitation seesaw and droughts of 1302 to 1307 in Europe, Clim. Past, 16, 2343–2358, https://doi.org/10.5194/cp-16-2343-2020, 2020.
Cited by
5 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
|
|