Does ChatGPT Play a Double-Edged Sword Role in the Field of Higher Education? An In-Depth Exploration of the Factors Affecting Student Performance

Abstract

The application of generative artificial intelligence in the field of education has been receiving increasing attention, with the performance of chatbot ChatGPT being particularly prominent. This study aims to explore in depth the performance impact on higher education students utilizing ChatGPT. To this end, we conducted a survey on 448 university students and employed the partial-least squares (PLS) method of structural equation modeling for data analysis. The results indicate that all eight hypothetical paths posited in this study were supported, and surprisingly, the hypothesis that technology characteristics have a direct effect on performance impact was supported. Moreover, the study found that overall quality is a crucial factor determining performance impact. Overall quality indirectly affects performance impact through task-technology fit, technology characteristics, and compatibility, among which the mediating effect of compatibility is most significant, followed by technology characteristics. This study offers practical recommendations for students on the proper use of ChatGPT during the learning process and assists developers in enhancing the services of the ChatGPT system.