Sequential Labelling and DNABERT For Splice Site Prediction in Homo Sapiens DNA

Abstract

Abstract Background Gene prediction on DNA has been conducted using various deep learning architectures to discover splice sites to locate intron and exon regions. However, recent predictions are carried out with models trained with a sequence which has a splice site in the middle. This case eliminates the possibility of multiple splice sites in a single sequence. Results This research proposes a sequential labelling model to predict splice sites regardless of their position in a sequence. A sequential labelling model named DNABERT-SL is developed on pre-trained DNABERT-3. DNABERT-SL is benchmarked against the latest sequential labelling model for mutation type and location prediction based on BiLSTM and BiGRU. While achieving F1 scores above 0.8 on validation data, BiLSTM, BiGRU, and DNABERT-SL perform poorly on test data as indicated by their respective low F1 scores (0.498 ± 0.184, 0.6 ± 0.123, 0.532 ± 0.245). Conclusions DNABERT-SL model cannot distinguish nucleotides acting as splice sites from normal ones. Principal component analysis on token contextual representation produced by DNABERT-SL shows that the representation is not optimal for distinguishing splice site tokens from non-splice site tokens. Splice site motif observation conducted on test and training sequences shows that an arbitrary sequence with GT-AG motif can be both splice sites in some sequences and normal nucleotides in others.