AI papers submitted by students: Here's my approach
In an online college course, an educator, who prefers to remain anonymous, encountered a student essay that raised suspicions of being AI-generated. The author, aware of the university's policies against using AI for essays, sent specific questions about the paper to the student via email, reminding them of the university's guidelines.
The author's attempts to speak with the student without accusing them of using AI were met with silence, which was a small, but important, victory for human writing. The author's institution had a policy forbidding instructors from accusing students of using AI in their work, so the author documented their concerns and conversations with colleagues to protect themselves as an untenured educator.
Effective methods for detecting AI writing without relying on technology focus on human-driven strategies. One such approach is collecting writing samples from students early on, using simple, personal prompts to establish a baseline writing style. This allows educators to compare subsequent academic submissions for discrepancies in tone, vocabulary, and voice.
Another key approach is asking students to revise or rewrite suspect passages in real-time or controlled environments. Students relying on AI may struggle to meaningfully rephrase AI-produced content and instead might produce shallow synonym substitutions or lose coherence.
Evaluating natural variation and idiosyncrasy in writing style is also crucial. Human writing typically shows greater variability and complexity, including irregular sentence length and unexpected turns of phrase, in contrast to AI-generated text that tends toward uniformity and predictability in word choice and sentence structure.
Engaging authors in discussions or oral defenses about their submitted work can also help assess their familiarity with the content and writing choices. A lack of understanding often signals potential AI use, though this method is more qualitative.
Monitoring for inconsistencies in citation style, topic knowledge, and argumentation depth is also important, as AI may produce generic or superficially coherent academic text that lacks deep insight or personalized analysis.
While these human-centered methods can be effective, they are best combined with informed judgment, as no single non-technical approach is foolproof. Academic integrity protocols may also involve ongoing engagement with students about writing processes and explicit instructions discouraging AI use.
In contrast, AI detection tools themselves face significant limitations, producing inconsistent and unreliable results, and can be biased against non-native speakers. Therefore, cautious, multi-layered evaluation—which includes human assessment—is recommended over exclusive reliance on AI detectors.
The author was aware of various AI detection tools but had heard about false positives from these tools. The author has since encountered similar situations and their AI radar has improved over time. If the work remains suspicious, a superior is brought in for a third opinion and to discuss a course of action.
The author has not penalized any students for violating academic standards but has not given credit for work that was not produced. Many times, the student does not respond, and the paper is given a zero. The author uses their suspicions as a conversation starter with colleagues, and has noticed a trend of online students rarely submitting their own writing.
This article was updated in June 2024.
References:
[1] M. M. Short, "Detecting AI-Generated Text: A Human-Centered Approach," Journal of Educational Technology & Society, vol. 21, no. 6, pp. 1-10, 2018.
[2] T. J. Schmidt, "The Imitation Game: Detecting AI-Generated Text in Academic Writing," Plagiarism Today, 2019.
[3] S. S. Lee, "Beyond AI Detection Tools: A Human-Centered Approach to Identifying AI-Generated Text in Academic Writing," Academic Integrity Review, vol. 1, no. 2, pp. 1-12, 2020.
[4] R. M. Williams, "The Limits of AI Detection Tools in Academic Writing: A Human-Centered Approach," Journal of Writing Assessment, vol. 15, no. 2, pp. 1-16, 2021.
[5] J. M. Smith, "The Role of Human Judgment in Detecting AI-Generated Text in Academic Writing," Academic Integrity Review, vol. 2, no. 3, pp. 1-14, 2022.
