Abstract
AbstractGenerative pre-trained models have achieved remarkable success in various domains such as natural language processing and computer vision. Specifically, the combination of large-scale diverse datasets and pre-trained transformers has emerged as a promising approach for developing foundation models. Drawing parallels between linguistic constructs and cellular biology — where texts comprise words, similarly, cells are defined by genes — our study probes the applicability of foundation models to advance cellular biology and genetics research. Utilizing the burgeoning single-cell sequencing data, we have pioneered the construction of a foundation model for single-cell biology, scGPT, which is based on generative pre-trained transformer across a repository of over 33 million cells. Our findings illustrate that scGPT, a generative pre-trained transformer, effectively distills critical biological insights concerning genes and cells. Through the further adaptation of transfer learning, scGPT can be optimized to achieve superior performance across diverse downstream applications. This includes tasks such as cell-type annotation, multi-batch integration, multi-omic integration, genetic perturbation prediction, and gene network inference. The scGPT codebase is publicly available athttps://github.com/bowang-lab/scGPT.
Publisher
Cold Spring Harbor Laboratory
Reference100 articles.
1. Cell-free gene expression: an expanded repertoire of applications;In: Nature Reviews Genetics,2020
2. Sebastian Preissl , Kyle J Gaulton , and Bing Ren . “Characterizing cis-regulatory elements using single-cell epigenomics”. In: Nature Reviews Genetics (2022), pp. 1–23.
3. Temporal modelling using single-cell tran-scriptomics;In: Nature Reviews Genetics,2022
4. Lineage tracing meets single-cell omics: opportunities and challenges;In: Nature Reviews Genetics,2020
5. HCA. HCA DCP. https://data.humancellatlas.org/. Online; accessed 12 April 2023. 2023.