The Unseen Pitfalls of A.I. Detection Tools
Janelle Shane, a scientist with a doctoral degree in engineering, has been a voice of authority on A.I. for nearly a decade. She’s been working on complex, custom light control systems for organizations like NASA, and in her spare time, she’s been dissecting the hype and realities of A.I. through her blog, AI Weirdness.
In a recent post, she put A.I. detection tools under the microscope, and the results were far from reassuring.
A.I. Detection Tools: A False Sense of Security
Shane highlighted a study from Stanford University that revealed a startling truth about A.I. detection products. These tools, designed to identify text written by language models like ChatGPT, are failing miserably at their job. They often misidentify A.I.-generated content as human-written and are heavily biased against non-native English-speaking writers.
In essence, A.I. detection tools are not only failing to identify A.I.-generated content but are also unfairly flagging content written by non-native English speakers. The irony is that A.I. thinks other A.I. is human, and it thinks humans writing in a language they’re not proficient in are A.I.
The Unintended Consequences of A.I. Detection
Shane’s blog post also delved into the implications of these findings. She pointed out that a student who uses A.I. to write or reword their essay is less likely to be flagged as a cheater than a student who never used A.I. at all.
She also highlighted how A.I. detectors often mislabel non-native English speakers’ writing as A.I.-generated. The study found that tools like Originality.ai, Quill.org, and Sapling GPT were misclassifying writing by non-native English speakers as A.I.-generated 48%-76% of the time, compared to 0%-12% for native speakers.
The Call to Action
Both Shane and the Stanford professors are calling for action. They caution against the use of GPT detectors in evaluative or educational settings, especially when assessing the work of non-native English speakers. The high rate of false positives for non-native English writing samples identified in the study highlights the potential for unjust consequences and the risk of exacerbating existing biases against these individuals.
In the end, Shane’s conclusion is clear and unequivocal: “Don’t use AI detectors for anything important.”
