Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models-Reference-Cited by-同舟云学术

Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models

Published:2022-12-20 Issue: Volume: Page:
ISSN:
Container-title:
language:
Short-container-title:

Author:

Kung Tiffany H.,Cheatham Morgan,Medenilla Arielle,Sillos Czarina,De Leon Lorie,Elepaño Camille,Madriaga Maria,Aggabao Rimel,Diaz-Candido Giezel,Maningo James,Tseng Victor,

Abstract

ABSTRACTWe evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.

Publisher

Cold Spring Harbor Laboratory

Reference21 articles.

1. Szegedy, C. , Vanhoucke, V. , Ioffe, S. , Shlens, J. & Wojna, Z. Rethinking the Inception Architecture for Computer Vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Preprint at https://doi.org/10.1109/cvpr.2016.308 (2016).

2. Zhang, W. , Feng, Y. , Meng, F. , You, D. & Liu, Q. Bridging the Gap between Training and Inference for Neural Machine Translation. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics Preprint at https://doi.org/10.18653/v1/p19-1426 (2019).

3. Bhatia, Y. , Bajpayee, A. , Raghuvanshi, D. & Mittal, H. Image Captioning using Google‘s Inception-resnet-v2 and Recurrent Neural Network. 2019 Twelfth International Conference on Contemporary Computing (IC3) Preprint at https://doi.org/10.1109/ic3.2019.8844921 (2019).

4. McDermott, M. B. A. et al. Reproducibility in machine learning for health research: Still a ways to go. Sci. Transl. Med. 13, (2021).

5. How to develop machine learning models for healthcare

Cited by 96 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. The emergence of compositionality in a brain-inspired cognitive architecture;Cognitive Systems Research;2024-08

2. Evaluation of ChatGPT as a diagnostic tool for medical learners and clinicians;PLOS ONE;2024-07-31

3. Automatic Feedback Generation on K-12 Students' Data Science Education by Prompting Cloud-based Large Language Models;Proceedings of the Eleventh ACM Conference on Learning @ Scale;2024-07-09

4. Generative AI: A Case Study of ChatGPT’s Impact on University Students’ Learning Practices;2024-07-01

5. Public attitudes toward chatgpt on twitter: sentiments, topics, and occupations;Social Network Analysis and Mining;2024-05-20