Comparative Eminence: Foundation versus Domain-Specific Model for Cardiac Ultrasound Segmentation

Abstract

AbstractImportanceA recently developed vision foundation model, “Segment Anything (SAM),” promises to segment any objects in images. However, the performance of SAM on clinical echocardiography images is yet to be investigated and compared against the domain-specific models.ObjectiveTo evaluate the performance of SAM on transthoracic echocardiography (TTE) and point-of-care ultrasound (POCUS) images.DesignSAM was fine-tuned on the training set of EchoNet-Dynamic (TTE) and then evaluated on datasets containing TTE and POCUS images.SettingMulti-center, retrospective cohort study.ParticipantsThis study used two publicly available datasets (EchoNet-dynamic, Stanford University and CAMUS, University Hospital of St Etienne). The Mayo Clinic dataset contains a sample of 99 non-duplicated patients (58 TTE and 41 POCUS).Intervention/Exposurenot applicable.Main Outcomes and MeasuresModel segmentation performance: Dice similarity coefficient (DSC).ResultsFine-tuned SAM had promising frame-level performance (SAM vs. EchoNet: DSC 0.911 ± 0.045 vs. 0.915 ± 0.047, p<0.0001), and consistent performance on the external datasets including TTE (Mayo Clinic: DSC 0.902 ± 0.032 vs. 0.893 ± 0.090, p<0.0001, CAMUS-A4C: DSC 0.897 ± 0.036 vs. 0.850 ± 0.097, p<0.0001, CAMUS-A2C: DSC 0.891 ± 0.040 vs. 0.752 ± 0.196, p<0.0001) and POCUS (DSC 0.857 ± 0.047 vs. 0.667 ± 0.279, p<0.0001).Conclusions and RelevancePromising segmentation performance was observed after fine-tuning the SAM model on TTE. The strong generalization capability of SAM can facilitate the development of AI applications in cardiac ultrasound with less manual data curation.Key pointsQuestionWhat is the comparative performance of fine-tuned Segment Anything Model (SAM) against domain-specific segmentation model on transthoracic echocardiography (TTE) and point-of-care ultrasound (POCUS)?FindingsFine-tuned SAM had excellent performance on EchoNet dataset (SAM vs. EchoNet: DSC 0.911 ± 0.045 vs. 0.915 ± 0.047, p<0.0001) and generalized well on external datasets containing TTE (Mayo TTE: DSC 0.902 ± 0.032 vs. 0.893 ± 0.090, p<0.0001) and POCUS (DSC 0.857 ± 0.047 vs. 0.667 ± 0.279, p<0.0001).MeaningThe generalization capability of SAM can facilitate the development of AI applications in echocardiography and POCUS with minimal expert data curation.