STimage:robust, confident and interpretable models for predicting gene markers from cancer histopathological images

Abstract

AbstractSpatial transcriptomic (ST) data enables us to link tissue morphological features with thousands of unseen gene expression values, opening a horizon for breakthroughs in digital pathology. Models to predict the presence/absence, high/low, or continuous expression of a gene using images as the only input have a huge potential clinical applications, but such models require improvements in accuracy, interpretability, and robustness. We developed STimage models to estimate parameters of gene expression as distributions rather than fixed data points, thereby allowing for the essential quantification of uncertainty in the predicted results. We assessed aleatoric and epistemic uncertainty of the models across a diverse range of test cases and proposed an ensemble approach to improve the model performance and trust. STimage can train prediction models for one gene marker or a panel of markers and provides important interpretability analyses at a single-cell level, and in the histopathological annotation context. Through a comprehensive benchmarking with existing models, we found that STimage is more robust to technical variation in platforms, data types, and sample types. Using images from the cancer genome atlas, we showed that STimage can be applied to non-spatial omics data. STimage also performs better than other models when only a small training dataset is available. Overall, STimage contributes an important methodological advance needed for the potential application of spatial technology in cancer digital pathology.