Predicting gene and protein expression levels from DNA and protein sequences with Perceiver

Abstract

AbstractBackground and ObjectiveThe functions of an organism and its biological processes result from the expression of genes and proteins. Therefore quantifying and predicting mRNA and protein levels is a crucial aspect of scientific research. Concerning the prediction of mRNA levels, the available approaches use the sequence straddling the Transcription Start Site (TSS) as input to neural networks. The State-of-the-art models (e.g., Xpresso and Basenjii) predict mRNA levels exploiting Convolutional (CNN) or Long Short Term Memory (LSTM) Networks. However, CNN prediction depends on convolutional kernel size, and LSTM suffers from capturing long-range dependencies in the sequence. Concerning the prediction of protein levels, as far as we know, there is no model for predicting protein levels by exploiting the gene or protein sequences.MethodsHere, we exploit a new model type (called Perceiver) for mRNA and protein level prediction, exploiting a Transformer-based architecture with an attention module to attend to long-range interactions in the sequences. In addition, the Perceiver model overcomes the quadratic complexity of the standard Transformer architectures. This work’s contributions are 1. DNAPerceiver model to predict mRNA levels from the sequence straddling the TSS; 2. ProteinPerceiver model to predict protein levels from the protein sequence; 3. Protein&DNAPerceiver model to predict protein levels from TSS-straddling and protein sequences.ResultsThe models are evaluated on cell lines, mice, glioblastoma, and lung cancer tissues. The results show the effectiveness of the Perceiver-type models in predicting mRNA and protein levels.ConclusionsThis paper presents a Perceiver architecture for mRNA and protein level prediction. In the future, inserting regulatory and epigenetic information into the model could improve mRNA and protein level predictions. The source code is freely available athttps://github.com/MatteoStefanini/DNAPerceiverGraphical AbstractHighlightsPredicting mRNA and protein levels from DNA and protein sequences is crucial in clinical applications.A transformer-based architecture with asymmetric attention (Perceiver) is exploited for mRNA and protein level prediction.The Perceiver architecture attends to longer range interactions compared to Transformer, CNN, and LSTM.The proposed model achieves state-of-the-art performance for mRNA level prediction.To the best of our knowledge, the protein level prediction task is addressed.The proposed model is tested on glioblastoma and lung cancer tissues.