AI Detection's High False Positive Rates and the Psychological and Material Impacts on Students

Abstract

This chapter, per the authors, explains the inherent impossibility of “AI detection,” and explores the material and psychological impacts of AI detection false positives on students. A small corpus study is presented demonstrating much higher than advertised rates of false positives across a range of popular “AI detection” tools. Based on this study along with news reports and first-person testimony from affected students, the chapter presents the possibility that neurodivergent writers, along with L2 writers, are more likely to be impacted by false positives. Given the current rates of mental health challenges on college campuses and the likelihood of a disproportionate impact on students who already face marginalization, the use of these AI detection tools is argued to be unethical. The chapter closes with recommendations for writing teachers.