Validation of the Seattle Suprastomal Safety Score (5S): A Novel Measure in Pediatric Tracheostomy-Dependent Patients

Abstract

Objective Suprastomal collapse and granulation are common sequelae of pediatric tracheostomy. We present the first measure of suprastomal obstructive pathology, the Seattle Suprastomal Safety Score (5S), an instrument with 2 domains: collapse and granulation. Study Design Cross-sectional repeated testing survey. Setting Electronic survey. Methods A library of images was assembled from still pictures of the suprastomal area in 50 patients who previously underwent trachea-bronchoscopy at a quaternary children’s hospital. Five pediatric otolaryngologists and 2 pediatric pulmonologists reviewed the images in random, blinded fashion and provided 5S scores. Participants repeated this process 2 to 4 weeks later. Interrater agreement was calculated with an intraclass correlation coefficient (ICC) with a 2-way random-effects model and Fleiss’s κ. Intrarater agreement was measured with an ICC using a 2-way mixed-effects model as well as with test-retest correlations using Spearman rank coefficient. All measures were performed separately on collapse and granulation domains. Results ICC for interrater agreement was 0.88 (95% CI, 0.82-0.93) for collapse and 0.97 (95% CI, 0.96-0.98) for granulation, indicating almost perfect agreement. Fleiss’s κ demonstrated moderate agreement for collapse and almost perfect agreement for granulation. ICC for intrarater agreement was 0.95 (95% CI, 0.93-0.97) and 0.99 (95% CI, 0.98-0.99) for collapse and granulation, respectively, indicating almost perfect agreement. Spearman rank correlation for test-retest demonstrated substantial agreement for collapse and almost perfect agreement for granulation. Conclusion The 5S demonstrates excellent interrater and intrarater agreement, making it highly reliable as a novel measure of suprastomal collapse and granulation in tracheostomy-dependent pediatric patients.