Inter- and Intra-Rater Level of Agreement in Ultrasonic Video Grading of Venous Gas Emboli

Abstract

INTRODUCTION: This study aimed to evaluate whether a short familiarization session is sufficient for individuals with no prior experience of sonography to both reliably and consistently evaluate the prevalence of venous gas emboli (VGE) from precordial ultrasonic videos.METHODS: A total of 10 adults with no prior experience of sonography were introduced to the Eftedal-Brubakk 6-grade scale and were shown 6 video sequences, each of a maximum of 10 heartbeats, representing each grading level. Thereafter, they independently evaluated the prevalence of VGE in 70 ultrasonic videos before and after a 14-d interval (test-retest; intra-rater), with these being compared to an experienced sonographer’s grading (inter-rater).RESULTS: A significant inter-rater level of agreement was found between the naïve and experienced sonographers’ bubble grading both during the first (W = 0.945) and second (W = 0.952) round of bubble evaluation. The naïve observers’ evaluations were on average 79% (range: 61–95%) and 75% (range: 48–95%) in complete agreement with the experienced sonographer’s gradings, while the level of agreement was 99% and 98% within 1 grade unit. There was a significant intra-rater level of agreement (κ = 0.845) during the test-retest series, with a mean percentage level of agreement of 87% (range: 72–93%).CONCLUSION: This study demonstrates that a short familiarization session enables individuals with no prior sonography experience to consistently evaluate VGE prevalence from precordial ultrasonic videos.Elia A, Ånell R, Eiken O, Grönkvist M, Gennser M. Inter- and intra-rater level of agreement in ultrasonic video grading of venous gas emboli. Aerosp Med Hum Perform. 2022; 93(1):54–57.