Evaluation of cell-cell interaction methods by integrating single-cell RNA sequencing data with spatial information

Abstract

Abstract Background Cell-cell interactions are important for information exchange between different cells, which are the fundamental basis of many biological processes. Recent advances in single-cell RNA sequencing (scRNA-seq) enable the characterization of cell-cell interactions using computational methods. However, it is hard to evaluate these methods since no ground truth is provided. Spatial transcriptomics (ST) data profiles the relative position of different cells. We propose that the spatial distance suggests the interaction tendency of different cell types, thus could be used for evaluating cell-cell interaction tools. Results We benchmark 16 cell-cell interaction methods by integrating scRNA-seq with ST data. We characterize cell-cell interactions into short-range and long-range interactions using spatial distance distributions between ligands and receptors. Based on this classification, we define the distance enrichment score and apply an evaluation workflow to 16 cell-cell interaction tools using 15 simulated and 5 real scRNA-seq and ST datasets. We also compare the consistency of the results from single tools with the commonly identified interactions. Our results suggest that the interactions predicted by different tools are highly dynamic, and the statistical-based methods show overall better performance than network-based methods and ST-based methods. Conclusions Our study presents a comprehensive evaluation of cell-cell interaction tools for scRNA-seq. CellChat, CellPhoneDB, NicheNet, and ICELLNET show overall better performance than other tools in terms of consistency with spatial tendency and software scalability. We recommend using results from at least two methods to ensure the accuracy of identified interactions. We have packaged the benchmark workflow with detailed documentation at GitHub (https://github.com/wanglabtongji/CCI).