Defining and Assessing the Reproducibility of Crohn’s Disease Endoscopic Lesions: A Delphi-like Method from the GETAID

Abstract

Abstract Background and Aims Defining and assessing the reproducibility of Crohn’s disease [CD] endoscopic lesions is essential in assessing endoscopic healing. Methods Twelve endoscopic CD experts from the GETAID defined aphthoid erosions [AE], superficial ulcerations [SU], deep ulcerations [DU], stenosis, and fistulas according to a Delphi-like method. Thirty different GETAID physicians declared if they found acceptable each definition. Intra- and inter-observer agreements were investigated using 100 videos with one tagged specific lesion [AE, SU, DU, or sham lesion] read by 15 independent endoscopists at baseline and 1 month later in a randomised order. Video quality was determined by an external reader. According to kappa estimate [κ ±standard error], intra or inter-observer agreement was qualified as ‘moderate’ [0.4–0.6], ‘substantial’ [0.6–0.8], or ‘almost perfect’ [0.8–1.0]. Results Among 30 different experts, 83% to 97% found acceptable the definitions retrieved from the Delphi-like method. Intra-observer κ was 0.717 [±0.019] for SU, 0.681 [±0.027] for AE, 0.856 [±0.014] for DU, showing ‘substantial’ agreement. It was 0.801 [±0.016] for any ulceration [DU or SU]. There was a high variability across readers from ‘moderate’ to ‘almost perfect’ agreement. Inter-observer κ was 0.548 [±0.042] for SU, 0.554 [±0.028] for AE 0.694 [±0.041] for DU, and 0.705 [±0.042] for any ulceration. Inter-observer agreement increased when reading the 53 high-quality videos: 0.787 [±0.064] [p = 0.001], 0.607 [±0.043] [p = 0.001], and 0.782 [±0.064][p = 0.001] for DU, AE, and any ulceration, respectively. Conclusions Despite variable intra-agreement level across readers, the GETAID definitions for CD endoscopic lesions provided ‘substantial’ inter-observer agreements, especially in case of high-quality videos.