AI-Generated Content Is Undetectable

After several high-profile failures of AI detection tools (falsely accusing students of using AI, producing inconsistent results on the same text), a counter-narrative emerged claiming that AI detection is fundamentally impossible and that AI text is completely indistinguishable from human writing.

The truth is in the middle. Current AI detection tools are unreliable enough that they should not be used as sole evidence of AI authorship, especially in high-stakes contexts like academic misconduct cases. They produce both false positives (flagging human text as AI) and false negatives (missing AI text). However, AI-generated text does have statistical patterns that differ from human writing. Longer texts are easier to detect than short ones. Heavily edited AI text is harder to detect than raw output.

Watermarking techniques (embedding invisible statistical patterns in AI output) are being developed and show promise, but are not yet widely deployed.

Saying "AI text is undetectable" and "AI detection tools are unreliable" are different claims. The second is currently true. The first overstates the case. AI text does have detectable statistical properties, and detection will likely improve. The practical takeaway: do not trust AI detectors as definitive proof, but do not assume AI text is perfectly camouflaged either.

OpenAI: AI text classifier (discontinued) - https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text

GPTZero: AI detection - https://gptzero.me/

Stanford: Watermarking language models - https://arxiv.org/abs/2301.10226