Uncovering the effects of model initialization on deep model generalization: A study with adult and pediatric chest X-ray images

Abstract

AbstractModel initialization techniques are vital for improving the performance and reliability of deep learning models in medical computer vision applications. While much literature exists on non-medical images, the impacts on medical images, particularly chest X-rays (CXRs) are less understood. Addressing this gap, our study explores three deep model initialization techniques: Cold-start, Warm-start, and Shrink and Perturb start, focusing on adult and pediatric populations. We specifically focus on scenarios with periodically arriving data for training, thereby embracing the real-world scenarios of ongoing data influx and the need for model updates. We evaluate these models for generalizability against external adult and pediatric CXR datasets. We also propose novel ensemble methods: F-score-weighted Sequential Least-Squares Quadratic Programming (F-SLSQP) and Attention-Guided Ensembles with Learnable Fuzzy Softmax to aggregate weight parameters from multiple models to capitalize on their collective knowledge and complementary representations. We perform statistical significance tests with 95% confidence intervals andp-values to analyze model performance. Our evaluations indicate models initialized with ImageNet-pretrained weights demonstrate superior generalizability over randomly-initialized counterparts, contradicting some findings for non-medical images. Notably, ImageNet-pretrained models exhibit consistent performance during internal and external testing across different training scenarios. Weight-level ensembles of these models show significantly higher recall (p<0.05) during testing compared to individual models. Thus, our study accentuates the benefits of ImageNet-pretrained weight initialization, especially when used with weight-level ensembles, for creating robust and generalizable deep learning solutions.Author SummaryIn this research, we actively explore various techniques for optimal initialization of deep learning models for analyzing medical images such that the resulting models are generalizable and also demonstrate high performance. Generalizability is an area of significant importance. It is often ignored in favor of the model achieving high performance at the cost of maintaining on previously unseen, i.e. external data, that may also be out-of-distribution. This may result in the classifier performing inadequately thereby reducing its value. We demonstrate that unlike general-purpose images, such as those found in ImageNet collection, medical images, such as chest X-rays (CXRs), are different in their visual characteristics. We show that counter to previously reported results using non-medical images, ImageNet pre-trained models trained on medical images, in fact, converge sooner and generalize better than randomly initialized models. We compare three distinct model initialization methods using internal adult CXR data to train the models which are subsequently tested on external CXR images for both adult and pediatric populations. Additionally, we consolidate several of these models into an ‘ensemble’ to demonstrate that they achieve a more accurate identification of relevant cases during both internal and external testing. Therefore, our work underscores the promising potential of employing ImageNet-pretrained models for medical imagess and merging them into ensembles, aiming to enhance the reliability of AI in medical image analysis.