The Use of ChatGPT in Source-Based Writing Tasks

Abstract

AbstractChatGPT, a chatbot based on a Generative Pre-trained Transformer model, can be used as a teaching tool in the educational setting, providing text in an interactive way. However, concerns point out risks and disadvantages, as possible incorrect or irrelevant answers, privacy concerns, and copyright issues. This study aims to categorize the strategies used by undergraduate students completing a source-based writing task (SBW, i.e., written production based on texts previously read) with the help of ChatGPT and their relation to the quality and content of students’ written products. ChatGPT can be educationally useful in SBW tasks, which require the synthesis of information from a text in response to a prompt. SBW requires mastering writing conventions and an accurate understanding of source material. We collected 27 non-expert users of ChatGPT and writers (Mage = 20.37; SD = 2.17). We administered a sociodemographic questionnaire, an academic writing motivation scale, and a measure of perceived prior knowledge. Participants were given a source-based writing task with access to ChatGPT as external aid. They performed a retrospective think-aloud interview on ChatGPT use. Data showed limited use of ChatGPT due to limited expertise and ethical concerns. The level of integration of conflicting information showed to not be associated with the interaction with ChatGPT. However, the use of ChatGPT showed a negative association with the amount of literal source-text information that students include in their written product.