Unleashing the transformers: NLP models detect AI writing in education

Abstract

AbstractArtificial Intelligence (AI) has witnessed widespread application across diverse domains, with education being a prominent focus for enhancing learning outcomes and tailoring educational approaches. Transformer models, exemplified by BERT, have demonstrated remarkable efficacy in Natural Language Processing (NLP) tasks. This research scrutinizes the current landscape of AI in education, emphasizing the utilization of transformer models. Specifically, the research delves into the influence of AI tools facilitating text generation through input prompts, with a notable instance being the GPT-4 model developed by OpenAI. The study employs pre-trained transformer models to discern whether a given text originates from AI or human sources. Notably, BERT emerges as the most effective model, fine-tuned using a dataset comprising abstracts authored by humans and those generated by AI. The outcomes reveal a heightened accuracy in distinguishing AI-generated text. These findings bear significance for the educational realm, suggesting that while endorsing the use of such tools for learning, vigilance is warranted to identify potential misuse or instances where students should independently develop their reasoning skills. Nevertheless, ethical considerations must be paramount when employing such methodologies. We have highlighted vulnerabilities concerning the potential bias of AI models towards non-native English speakers, stemming from possible deficiencies in vocabulary and grammatical structure. Additionally, users must ensure that there is no complete reliance on these systems to ascertain students' performance. Further research is imperative to unleash the full potential of AI in education and address ethical considerations tied to its application.