A foundation model for generalizable cancer diagnosis and survival prediction from histopathological images

Abstract

AbstractComputational pathology, utilizing whole slide image (WSI) for pathological diagnosis, has advanced the development of intelligent healthcare. However, the scarcity of annotated data and histological differences hinder the general application of existing methods. Extensive histopathological data and the robustness of self-supervised models in small-scale data demonstrate promising prospects for developing foundation pathology models. Due to the need for deployment, lightweight foundation models also need to be developed. In this work, we propose the BEPH (BEiT-based modelPre-training onHistopathological images), a general lightweight foundation model that leverages self-supervised learning to learn meaningful representations from 11 million unlabeled histopathological images. These representations are then efficiently adapted to various tasks, including 2 cancer patch-level recognition tasks, 3 cancer WSI-level classification tasks, and 6 cancer subtypes survival prediction tasks. Experimental results demonstrate that our model consistently outperforms several comparative models with similar parameters, even with limited training data reduced to 50%. Especially when the downstream structure is the same, the model can improve ResNet and DINO by up to a maximum increase of 8.8% and 7.2% (WSI level classification), and 6.44% and 3.28% on average (survival prediction), respectively. Therefore, BEPH offers a universal solution to enhance model performance, reduce the burden of expert annotations, and enable widespread clinical applications of artificial intelligence. The code and models can be obtained athttps://github.com/Zhcyoung/BEPH. And currently, online fine-tuning of WSI classification tasks is available for use onhttp://yulab-sjtu.natapp1.cc/BEPH.