Unraveling the Inner Workings of Massive Language Models-Reference-Cited by-同舟云学术

Unraveling the Inner Workings of Massive Language Models

Published:2024-08-30 Issue: Volume: Page:239-279
ISSN:2327-0411
Container-title:Advances in Computational Intelligence and Robotics
language:
Short-container-title:

Author:

Babu C. V. Suresh¹^ORCID,Anniyappa C. S. Akkash²,Sastha B. Dharma¹

Affiliation:

1. Hindustan Institute of Technology and Science, India

2. Sri Sivasubramaniya Nadar College of Engineering, India

Abstract

This study explores the evolution of language models, emphasizing the shift from traditional statistical methods to advanced neural networks, particularly the transformer architecture. It aims to understand the impact of these advancements on natural language processing (NLP). The study examines the core concepts of language models, including neural networks, attention, and self-attention mechanisms, and evaluates their performance on various NLP tasks. The findings demonstrate significant improvements in language modeling, especially in dialogue generation and translation. Despite these advancements, the study highlights the need to address ethical issues such as bias, fairness, privacy, and security for responsible AI deployment.

Publisher

IGI Global

Reference93 articles.

1. TensorFlow: A system for large-scale machine learning.;M.Abadi;12th USENIX Symposium on Operating Systems Design and Implementation (OSDI),2016

2. Adiwardana, D., Luong, M. T., So, D. R., Hall, J., Fiedel, N., Thoppilan, R., & Le, Q. V. (2020). Towards a human-like open-domain chatbot. arXiv preprint arXiv:2001.09977.

3. Bapna, A., Firat, O., Wu, Y., Chuang, J., Dyer, C., & Smith, N. A. (2020). Simple and effective multi-paragraph reading comprehension. arXiv preprint arXiv:2001.07676.

4. Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science

5. On the Dangers of Stochastic Parrots