LETSmix: a spatially informed and learning-based domain adaptation method for cell-type deconvolution in spatial transcriptomics

Abstract

ABSTRACTSpatial transcriptomics (ST) has revolutionized our understanding of gene expression patterns by incorporating spatial context. However, many ST technologies operate on heterogeneous cell mixtures due to limited spatial resolution. Current methods for cell-type deconvolution often underutilize spatial context information inherent in ST and the paired histopathological images, meanwhile neglect domain variances between ST and the reference single-cell RNA sequencing (scRNA-seq) data. To address these issues, we present LETSmix, a deep learning-based domain adaptation method trained on labelled pseudo-spots generated from scRNA-seq data, and mixed real-spots that are refined by a designed LETS filter leveraging correlations among neighboring spots with similar morphological features. The performance of LETSmix is demonstrated across three public ST datasets through comprehensive assessments, setting a new record among current state-of-the-art models. Our findings indicate that LETSmix accurately estimates the proportions of various cell types, and effectively maps them to the expected areas. The utilization of domain adaptation techniques enables LETSmix to achieve highly stable results when trained with different reference scRNA-seq datasets. Applications of LETSmix to diverse tissues, including the human dorsolateral prefrontal cortex, human pancreatic ductal adenocarcinoma, and mouse liver, showcase its robust performance and generalizability across different biological contexts.GRAPHICAL ABSTRACT