Using AI to Evaluate a Competency-Based Online Writing Course in Nursing

Abstract

Nursing education is transitioning from traditional teaching to competency-based education. Additionally, more nursing courses and programs are now offered online. Scholarly writing is a powerful strategy to teach effective communication and critical thinking, both core competencies for safe and effective nursing practice. However, teaching writing online to nursing students is challenging due to a lack of research evaluating best practices, faculty time constraints, and inconsistent writing assessment. Automated essay scoring systems using artificial intelligence (AI) provide new opportunities for efficient, reliable, and valid assessment of writing skills. We used a quasi-experimental design to investigate the impact of a 14-week fully online competency-based writing course on students’ self-efficacy, task value, and writing performance. The participants were master’s nursing students enrolled in an existing one-semester online competency-based writing course for healthcare professionals. An AI-powered writing assessment, IntelliMetric®, and the SAWSES self-efficacy survey were administered pre- and post-intervention. The results showed statistically significant gains in self-efficacy and writing performance with large effect sizes. This study addresses the gap in nursing education regarding the assessment of online, research-based writing interventions on students’ scholarly writing capacity. Recommendations include implementing a required scholarly writing course in all graduate-level nursing programs, scaffolding students’ competency development with the cognitive apprenticeship model, using best practices from composition research to inform online instruction, and employing AI-powered automated essay scoring to evaluate students’ writing progress and instructional efficacy.