Awareness and Attitudes of Chinese Medical Students Towards the Application of Large Language Models in Medicine: A Cross-Sectional Survey Study (Preprint)

Abstract

ChatGPT, as a large language model, is having a significant impact on various fields. As a pioneering field, medical education must quickly adapt and prepare for the upcoming changes. However, the acceptance of this technology differs across countries due to varying national conditions. Despite these differences, the development of AI applications promises to bring many benefits to medical education. Although there are still several limitations, gathering more feedback from different areas will help better control the upcoming risks.

This study aims to assess the awareness and attitudes of medical students in mainland China towards the application of Large Language Models (LLMs) in the field of medicine.

The study involves total 490 undergraduate and graduate students from several tertiary medical centers in mainland China. The questionnaire, consisting of 20 questions, was developed through discussions among three teachers with over 5 years of teaching experience. It covers four categories which are: background information, cognitive level, potential acceptance, and potential expectations, with each dimension containing 4-6 questions. The questionnaires were administered via the "Sojump" platform, and the survey was conducted online.

The survey results indicate male participants exhibit higher acceptance of LLMs, as evidenced by higher engagement on social media, usage of LLMs tools for academic writing, and translation of literature compared to female participants (P=.000, P=.000, P=.031, respectively). Additionally, graduate students are more likely inclined to gather information through academic journals than undergraduate students (P=.015). They also demonstrate greater proficiency in using LLMs tools for research-related tasks (academic writing P=.024, literature translation P=.006, respectively) and show higher interest in the clinical applications of LLMs (medical assistant P=.000, disease diagnosis and prognosis prediction based on deep learning models P=.001, respectively). Finally, almost all participants exhibit a positive attitude towards its application in the field of medicine, which showed a predominant tolerance to all questions from potential acceptance (PA) category and diminished anxiety about questions from potential concerns (PC) category.

This study reveals that Chinese medical students have an acceptable level of cognitive proficiency and a favorable attitude towards application of LLMs in medicine. Gender plays a role in influencing the application of LLMs, while male participants demonstrating a higher understanding and application level of LLMs tools compared to female counterparts. Educational background is also a crucial influencing factor, with graduate students exhibiting a deeper understanding of concepts of LLMs in academic and clinical application scenarios. Moreover, participants with LLMs knowledge background showed a more positive attitude towards the application of LLMs. Therefore, comprehensively assessing these background factors and attitudes can provide invaluable guidance for tailoring and optimizing medical education strategies that incorporate LLMs effectively.

None