Weakly supervised contrastive learning predicts tumor infiltrating macrophages and immunotherapy benefit in breast cancer from unannotated pathology images

Abstract

AbstractThe efficacy of immune checkpoint inhibitors is significantly influenced by the tumor immune microenvironment (TIME). RNA sequencing of tumor biopsies or surgical specimens can offer valuable insights into TIME, but its high cost and long turnaround time seriously restrict its utility in routine clinical examinations. Several recent studies have suggested that ultra-high resolution pathology images can infer cellular and molecular characteristics. However, studies on revealing TIME from pathology images are still limited.In this paper, we proposed a novel weakly supervised contrastive learning model to deduce tumor immune microenvironment features from whole slide images (WSIs) of H&E stained pathological sections. The high-resolution WSIs are split into tiles, and then contrastive learning is applied to extract features of each tile. After aggregating the features at the tile level, we employ weak supervisory signals to fine-tune the encoder for various downstream tasks. Comprehensive downstream experiments on two independent breast cancer cohorts and spatial transcriptomics data demonstrate that our computational pathological features accurately predict the proportion of tumor infiltrating immune cells, particularly the infiltration level of macrophages, as well as the immune subtypes and biomarker gene expression levels. These findings demonstrate that our model effectively captures pathological features beyond human vision, establishing a mapping relationship between cellular compositions and histological morphology, thus expanding the clinical applications of digital pathology images.