Spatial Immunophenotyping from Whole-Slide Multiplexed Tissue Imaging Using Convolutional Neural Networks

Abstract

AbstractThe multiplexed immunofluorescence (mIF) platform enables biomarker discovery through the simultaneous detection of multiple markers on a single tissue slide, offering detailed insights into intratumor heterogeneity and the tumor-immune microenvironment at spatially resolved single cell resolution. However, current mIF image analyses are labor-intensive, requiring specialized pathology expertise which limits their scalability and clinical application. To address this challenge, we developed CellGate, a deep-learning (DL) computational pipeline that provides streamlined, end-to-end whole-slide mIF image analysis including nuclei detection, cell segmentation, cell classification, and combined immuno-phenotyping across stacked images. The model was trained on over 750,000 single cell images from 34 melanomas in a retrospective cohort of patients using whole tissue sections stained for CD3, CD8, CD68, CK-SOX10, PD-1, PD-L1, and FOXP3 with manual gating and extensive pathology review. When tested on new whole mIF slides, the model demonstrated high precision-recall AUC. Further validation on whole-slide mIF images of 9 primary melanomas from an independent cohort confirmed that CellGate can reproduce expert pathology analysis with high accuracy. We show that spatial immuno-phenotyping results using CellGate provide deep insights into the immune cell topography and differences in T cell functional states and interactions with tumor cells in patients with distinct histopathology and clinical characteristics. This pipeline offers a fully automated and parallelizable computing process with substantially improved consistency for cell type classification across images, potentially enabling high throughput whole-slide mIF tissue image analysis for large-scale clinical and research applications.