Evaluating Large Language Models on Academic Literature Understanding and Review: An Empirical Study among Early-stage Scholars-Reference-Cited by-同舟云学术

Evaluating Large Language Models on Academic Literature Understanding and Review: An Empirical Study among Early-stage Scholars

Published:2024-05-11 Issue: Volume:55 Page:1-18
ISSN:
Container-title:Proceedings of the CHI Conference on Human Factors in Computing Systems
language:
Short-container-title:

Author:

Wang Jiyao¹^ORCID,Hu Haolong²^ORCID,Wang Zuyuan¹^ORCID,Yan Song¹^ORCID,Sheng Youyu¹^ORCID,He Dengbo²^ORCID

Affiliation:

1. Robotics and Autonomous Systems Thrust, The Hong Kong University of Science and Technology (Guangzhou), China

2. Intelligent Transportation Thrust, The Hong Kong University of Science and Technology (Guangzhou), China

Funder

Guangzhou Municipal Science and Technology Project

Guangzhou Science and Technology Program City-University Joint Funding Project

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3613904.3641917

Reference68 articles.

1. Ian L Alberts, Lorenzo Mercolli, Thomas Pyka, George Prenosil, Kuangyu Shi, Axel Rominger, and Ali Afshar-Oromieh. 2023. Large language models (LLM) and ChatGPT: what will the impact on nuclear medicine be?European Journal of Nuclear Medicine and Molecular Imaging 50, 6 (2023), 1549–1552.

2. Concerns with the Usage of ChatGPT in Academia and Medicine: A Viewpoint

3. Ömer Aydın and Enis Karaarslan. 2022. OpenAI ChatGPT generated literature review: Digital twin in healthcare. Available at SSRN 4308687 (2022).

4. Stephen H Bach, Victor Sanh, Zheng-Xin Yong, Albert Webson, Colin Raffel, Nihal V Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Fevry, 2022. Promptsource: An integrated development environment and repository for natural language prompts. arXiv preprint arXiv:2202.01279 (2022).

5. On the Dangers of Stochastic Parrots