Foundation models for bioinformatics-Reference-Cited by-同舟云学术

Foundation models for bioinformatics

Published:2024-07-24 Issue: Volume: Page:
ISSN:2095-4689
Container-title:Quantitative Biology
language:en
Short-container-title:Quant. Biol.

Author:

Chen Ziyu¹²,Wei Lin¹²,Gao Ge¹²

Affiliation:

1. State Key Laboratory of Protein and Plant Gene Research School of Life Sciences Biomedical Pioneering Innovative Center (BIOPIC) & Beijing Advanced Innovation Center for Genomics (ICG) Center for Bioinformatics (CBI) Peking University Beijing China

2. Changping Laboratory Beijing China

Abstract

AbstractTransformer‐based foundation models such as ChatGPTs have revolutionized our daily life and affected many fields including bioinformatics. In this perspective, we first discuss about the direct application of textual foundation models on bioinformatics tasks, focusing on how to make the most out of canonical large language models and mitigate their inherent flaws. Meanwhile, we go through the transformer‐based, bioinformatics‐tailored foundation models for both sequence and non‐sequence data. In particular, we envision the further development directions as well as challenges for bioinformatics foundation models.

Funder

National Key Research and Development Program of China

Publisher

Wiley

Link

https://onlinelibrary.wiley.com/doi/pdf/10.1002/qub2.69

Reference80 articles.

1. BommasaniR HudsonDA AdeliE AltmanR AroraS vonArxS et al.On the opportunities and risks of foundation models.2021. Preprint at arXiv: 2108.07258.

2. ZhaoWX ZhouK LiJ TangT WangX HouY et al.A survey of large language models.2023. Preprint at arXiv: 2303.18223.

3. VaswaniA ShazeerN ParmarN UszkoreitJ JonesL GomezAN et al.Attention is all you need.2017. Preprint at arXiv: 1706.03762.

4. UszkoreitJ.Transformer: a novel neural network architecture for language understanding. Google Research Blog.2017.

5. BahdanauD ChoK BengioY.Neural machine translation by jointly learning to align and translate.2014. Preprint at arXiv: 1409.0473.