Feed the 1787 US Constitution into an AI detector and it will tell you, with high probability: written by an AI. A text that predates ChatGPT by 236 years.

This isn’t a cherry-picked fluke — it’s a clean experiment. GPTZero’s own founder, Edward Tian, explained the cause himself: the Constitution is a text that has been fed into the language models over and over. The detector sees in it exactly what it sees in real AI. Only here, wrongly.

And that’s the whole problem. An AI detector does not measure whether a text came from an AI. It measures something else — and mistakes it for AI authorship.

What a detector actually measures

A plagiarism scanner puts the source next to your text. You can check: here’s the copied paragraph, there’s the original. That’s verifiable, and refutable.

An AI detector can’t do that. It has no source to point to. It measures how predictable a text is — in technical terms, its perplexity. Language models produce likely, smooth, expected sequences of words. So the reverse gets treated as suspicious: write smoothly and predictably, and to the detector you look like a machine.

That’s the entire trick. And it’s the entire flaw.

Because who writes smoothly and predictably? Not just AI. Also people who learned the language as a second one and lean on safe, standard phrasing. People who write plainly and clearly. Some neurodivergent writers. The detector can’t tell these people from an AI, because it never looked for AI — only for smoothness.

The proof that it’s about language, not AI

A study from Stanford University (Liang et al., published in Patterns by Cell Press, 10 July 2023) tested seven commercial detectors on real, guaranteed-human texts.

On essays by US 8th-graders, the error rate was near zero. On TOEFL essays — written by people for whom English is a foreign language — the same detectors wrongly classified an average of 61.22% as AI. Almost 98% of the essays were flagged by at least one tool, one in five unanimously by all seven.

Then comes the part that explains everything. The researchers had GPT-4 polish the TOEFL essays linguistically — richer word choice, nothing else. The false AI classification dropped from 61.22% to 11.77%.

Same content, same people, same detectors. Just richer language. If a better vocabulary turns an “AI text” into a “human text”, then the detector isn’t measuring AI. It’s measuring language proficiency. And penalising the lower kind.

What happens when you test many tools side by side

The broadest independent study to date comes from Weber-Wulff, Anohina-Naumeca, Foltýnek and colleagues (International Journal for Educational Integrity 19:26, peer-reviewed, 25 December 2023). They tested 14 detectors on 54 documents of known origin — 756 individual tests.

The result: not a single tool reached 80% accuracy, only five made it over 70. False-positive rates ranged from 0% (Turnitin in this test) to 50% (GPTZero in this test). Around 20% of the genuine AI texts were attributed to humans — after light editing, fully half.

The authors write that the systems “should not be used in academic settings”. And they name the real crux: unlike a plagiarism scanner, there is no source document. A student accused on this basis alone would have “no possibility for a defence”.

You cannot refute what was never evidenced.

The most honest witness: OpenAI itself

If anyone should be able to detect AI text, it’s the maker of ChatGPT. OpenAI tried — with its own “AI Text Classifier”, released in January 2023.

After barely six months, OpenAI took it down. The reason is in their own update note: due to its low rate of accuracy. The tool caught only 26% of AI texts and wrongly flagged 9% of human texts as AI. On short texts under 1,000 characters it was, in their words, “very unreliable”.

The maker who built the model could not reliably detect its output — and was honest enough to pull the product.

What the providers put in their own fine print

The most revealing material isn’t in the marketing — it’s in the FAQs.

Turnitin, the market leader in education, cites a document-wide false-positive rate of “under 1%”. Read on, and that figure only applies to documents with over 20% AI content; sentence by sentence, the rate is around 4%; below 20% the tool shows no figure at all, just an asterisk. And Turnitin itself writes that the result “should not be used as the sole basis for action”.

GPTZero writes in its own FAQ that its results “should not be used to punish students”. And: the training dataset is mostly English prose written by adults. That’s exactly what explains the bias.

That’s the through-line: the providers themselves warn against precisely the use — punishment on suspicion — that teachers put the tools to.

In fairness there’s a counter-finding, and it belongs here: Turnitin published its own study (26 October 2023) claiming no statistically significant disadvantage for English-language learners above 300 words. That contradicts Stanford only in part — for long texts. For short ones, Turnitin itself concedes higher error rates. And it’s a witness in its own cause. Short tasks, summaries, two-paragraph answers are everyday coursework — exactly where the numbers are worst.

Why a small percentage is still dangerous

“Under 1%” sounds reassuring. Scale it up.

Vanderbilt University submitted around 75,000 papers in 2022. Arithmetically — not measured — Turnitin’s own “under 1%” would yield roughly 750 cases in which a human is wrongly flagged. Whether those cases occurred, the arithmetic doesn’t say. But Vanderbilt drew the conclusion and switched the detector off on 16 August 2023, stating it did not believe AI detection software was an effective tool that should be used.

At scale, a low error rate is not a small number. It’s a three-digit count of suspicions per year — each with a person behind it who did nothing.

The most important point first, and it’s in almost no summary: in Germany, the burden of proof lies with the university. Deception must be proven by the examination authority — and both the deception and the penalty must be set out in the examination regulations in advance. A detector score does not reverse that burden.

The most recent ruling comes from the Administrative Court of Kassel (25 February 2026, ref. 7 K 2134/24.KS and 7 K 2515/25.KS). The court upheld penalties for AI use — but expressly did not rely on a detector, basing its finding instead on human-checkable indicators: a striking discrepancy between written and oral performance, over-repeated stock phrases. The rulings are not yet final (as of 24 July 2026); whether a higher court confirms them is open.

A practical guidance note from the Digitale Lehre Hub Niedersachsen (Baresel, Horn & Schorer, as of 24 February 2025) advises universities against using AI detectors. Their arguments: personal examination data (Art. 6 GDPR), the ban on purely automated decisions (Art. 22 GDPR), a possible classification as high-risk AI under the EU AI Act. In the cited urgent proceedings before the Munich Administrative Court, detector results counted only as one indicator, never as proof.

Some German universities draw the clearest conclusion and deliberately provide no AI detectors at all — as official policy, not an individual opinion.

What to do if a detector flags you wrongly

This is journalistic context, not legal advice for your specific case — that takes someone who knows the file. But the direction the evidence and the cited guidance point in can be named:

The score alone is not proof. It states a probability, not a verifiable source. Ask for the basis: which tool, which version, which figure, measured against what? A “high AI score” without those details is not evidence, it’s an assertion.

Show your process. This is the strongest counter-evidence — the one thing a detector precisely cannot provide: the version history of your document (Google Docs, Word, every cloud keeps one), drafts, notes, research tabs, sources. A real writing process leaves traces. A probability does not.

Name the bias if it applies to you. If German — or English — is not your first language, the Stanford study is your source: the same tools that wave native speakers through almost flawlessly flag non-native writers en masse. That’s not a mark against you, it’s a known, measured design flaw in the tools.

Know where the burden lies. In Germany the university must prove the deception, not you your innocence. If it comes to that, this belongs in the hands of your university’s legal advice service, a specialist lawyer, or the data protection officer.

The honest counterpoint

So this doesn’t become comfortable one-sidedness: there are two things in the detectors’ favour — and both have to be said.

First, humans are even worse at spotting AI text. A study on academic writing (Cheng et al., 2025) found human reviewers scoring 19% — chance level. The detectors beat them clearly. Only the same study also measured a 72.2% false-positive rate on purely human text. “Better than humans at detecting” and “useless as proof” don’t actually contradict each other.

Second: detection at the source works. Google DeepMind demonstrated a text watermark in Nature (23 October 2024) — SynthID-Text — that reliably marks AI output without measurably lowering text quality, tested on around 20 million Gemini responses. The difference is decisive: the signal is built in at generation, not guessed at afterwards. That’s exactly why regulation is betting on it too: from 2 August 2026, Article 50 of the EU AI Act (per the European Commission’s FAQ) requires providers of generative AI to mark their output in a machine-readable way — with a transition period for existing systems until 2 December 2026. (That’s the legal position per the primary source, not legal advice; check the status as of your reading date.)

So the way forward is marking at the source, not guessing at the result. That politics chooses this path is a quiet admission: the after-the-fact detector doesn’t solve the problem.

The one line to take away

An AI detector is a conversation starter, not a verdict. It may prompt a question — “tell me how this text came about”. It may never have the final word. Not because that’s a nice attitude, but because technically it can do nothing else: it measures smoothness, states a probability, and lays no source beside it. Anyone who bases a grade or an accusation on that has mistaken a hunch for proof.

And the Constitution of 1787 wasn’t one.

Sources

All checked on 24 July 2026. Figures apply as of the stated measurement date; detector values and the legal status may have changed since.

If something here looks outdated or wrong to you: get in touch.