A detector can be "99% accurate" and still wrongly flag dozens of honest students in a single school year. Both can be true at once, and the gap between them is where most of the confusion about AI detection lives.
So, are AI detectors accurate? Independent researchers have tested them many times since 2023. The answer depends heavily on the tool, the length of the text, whether anyone edited the AI output, and who wrote the human text. This guide walks through what those studies found, explains the terms that accuracy claims hide behind, and works through a 1,000-essay example so you can see how errors add up.
If you're new to how detectors make their guesses, start with our guide on how AI detectors work.
Are AI detectors accurate?
AI detectors are accurate in some conditions and unreliable in others. Independent studies show they do best on long, unedited AI text in English, and worst on short texts, AI drafts that people edited or paraphrased, and writing by second-language writers. Accuracy also varies widely from tool to tool. A detector score is useful evidence to look closer, but it isn't reliable enough to prove anything on its own.
What does "accurate" mean for an AI detector?
"Accurate" can mean several different things, and companies don't always say which one they mean. Here are the terms you'll see in research.
Term | Plain-English meaning | The question it answers |
|---|---|---|
Accuracy | Share of all texts classified correctly | How often is the detector right overall? |
Recall (sensitivity) | Share of AI texts that get flagged | How much AI writing does it catch? |
Precision | Share of flagged texts that really are AI | When it flags something, how often is it right? |
False positive rate | Share of human texts wrongly flagged as AI | How often does it accuse an honest writer? |
False negative rate | Share of AI texts that slip through as human | How often does it miss AI writing? |
The false positive rate is the one that matters most when a flag can lead to a misconduct case. A 2025 working paper by Brian Jabarian and Alex Imas at the University of Chicago makes the same point. Anyone using a detector, they write, must weigh two numbers, the false negative rate and the false positive rate.
How accurate are AI detectors in independent studies?
Results vary a lot, but the pattern across independent studies is consistent. Detectors handle plain AI output better than edited AI output, and some tools wrongly flag human writing far more than others.
The 14-tool study (2023)
In December 2023, Debora Weber-Wulff and seven co-authors published a test of 14 detection tools in the International Journal for Educational Integrity. They wrote test documents with known origins in several categories. Some were human-written, some were AI-written, and some were AI-written then edited by hand or by a paraphrasing tool.
No tool scored above 80% accuracy, and only five scored above 70%. Human-written English text was classified correctly 96% of the time on average. Unedited AI text was caught 74% of the time. Once people edited the AI text by hand, that fell to 42%. After machine paraphrasing, it fell to 26%. The authors concluded that the tools were "neither accurate nor reliable" and leaned toward calling text human.
Adversarial edits (2024)
In 2024, Mike Perkins and colleagues tested six detectors on 805 AI-written samples, publishing in the International Journal of Educational Technology in Higher Education. Average detection accuracy was 39.5% on the unmodified samples. When the researchers applied simple techniques to disguise the AI text, it fell to 17.4%. They also ran 10 human-written samples, and some tools flagged a few of them as AI.
Newer versus older AI text (2023)
In September 2023, Ahmed Elkhatat and colleagues in Qatar tested five tools on paragraphs about cooling towers. The tools did better on text from an older AI model than a newer one. On the human-written control samples, they gave inconsistent results, including false positives.
Second-language writers (2023)
James Zou's team at Stanford ran 91 TOEFL essays by non-native English speakers through seven detectors, publishing in Patterns in July 2023. On average, 61% of those human essays were wrongly flagged as AI. Essays by US eighth graders were classified almost perfectly.
Large benchmarks and newer tests (2024 and 2025)
The RAID benchmark, presented at ACL 2024 by Liam Dugan and colleagues at the University of Pennsylvania, tested 12 detectors on over 6 million AI-generated texts. It found detectors were easily fooled by deliberate attacks, by changes in how text was generated and by AI models they hadn't seen. It also noted that many detectors claim 99% accuracy or higher.
The Chicago working paper from September 2025 found a wide spread between tools. The best commercial detector they tested made almost no errors on medium and long passages. An open-source baseline did poorly across the board, and errors rose for most tools on shorter passages and on text run through "humanizer" tools. That paper hasn't been peer-reviewed yet, but it's a useful reminder that "AI detectors" aren't one thing.
Why can a "99% accurate" detector still flag innocent writers?
Because accuracy depends on how common AI writing is in the pile you're checking. This is the base rate problem, and it's easiest to see with numbers.
Say a teacher checks 1,000 essays, and 100 of them were really written with AI. The detector in this example (the figures are made up to show the math) catches 90% of AI essays and wrongly flags 2% of human ones.
Flagged as AI | Not flagged | Total | |
|---|---|---|---|
Really AI-written | 90 | 10 | 100 |
Really human-written | 18 | 882 | 900 |
Total | 108 | 892 | 1,000 |
Overall accuracy is 97.2%, which sounds great. But look at the 108 flagged essays. Eighteen of them are innocent. That's one in six flags pointing at a student who did nothing wrong. The precision here is 90 divided by 108, about 83%.
Now change one thing. Say only 20 of the 1,000 essays used AI. The detector still catches 18 of them. It also flags 2% of 980 human essays, which is about 20. Now roughly half of all flags are wrong, even though the detector itself didn't change.
Scale makes this worse. Jisc, the UK's education technology body, worked through a similar example in June 2025. An institution with 20,000 students, each taking 8 modules with 3 assessments, produces 480,000 assessments a year. Even a 1% false positive rate would mean about 4,800 false positives a year.
What makes AI detection less reliable?
Six things push error rates up, and they often show up together.
Short texts. A 2025 review by Kathleen Fraser and colleagues at the National Research Council Canada, in the Journal of Artificial Intelligence Research, found classifiers need roughly 120 to 200 words to reach their full potential.
Human editing. Edits by hand roughly halved detection in the 2023 14-tool study, from 74% to 42%.
Paraphrasing tools. Machine paraphrasing cut detection to 26% in the same study.
Machine translation. Human text written in another language and machine-translated to English was classified correctly about 20 percentage points less often.
Second-language writers. Their more constrained word choice can look predictable to a detector, as the Stanford study showed.
New AI models. The RAID benchmark found detectors struggled with models they hadn't been trained on.
Is AI detection more reliable than a human reader?
Not by default. People aren't a safe fallback. In its 2025 update, Jisc cites a study by Jon-Philippe Hyatt and colleagues that found a false positive rate of 1.3% for their AI detectors and 5% for human raters. It also cites work by Liz Hardie and colleagues showing that training people to spot AI writing made them catch more of it, but also raised their false positive rate.
The lesson is that neither a tool nor a hunch should decide a case alone. Jisc's advice is that decisions "should never be based solely on AI detection, whether by a tool or a person."
How should you judge an accuracy claim?
Ask what was tested, by whom, and which number is being reported. A single headline figure tells you very little.
Find out who ran the test. Independent studies by universities and journals count for more than a vendor's own page.
Look for the false positive rate on its own. If it's missing, the "accuracy" figure may be hiding it.
Check the text length. Results on 1,000-word essays don't apply to a 100-word discussion post.
Check for edited and paraphrased text. Real AI use is rarely a clean paste.
Check who wrote the human samples. If no second-language writers were included, the false positive rate may be too rosy for your class.
Ask what counted as a flag. The RAID authors showed results change a lot with the threshold, so they fixed each detector at a 5% false positive rate before comparing.
How should you use a detector score, then?
So, are AI detectors accurate enough to act on by themselves? The research says no. Treat a score as a signal to look closer, then look for evidence a detector can't give you. Check drafts and version history, compare the work with the writer's earlier writing, and talk with them about how they wrote it. If you're the writer, the same evidence is what will protect you.
A good detector also shows how sure it is. WrittenCheck gives an overall score, a likely range and a confidence level, plus a score for every sentence. A wide range or low confidence is a cue to rely even less on the number. You can paste a draft into our free AI detector to see which sentences read most like AI, then judge those sentences yourself.
Frequently asked questions
How accurate are AI detectors on average?
There's no single average worth quoting, because results depend on the tool, text length, language and editing. In a 2023 test of 14 tools, none topped 80% accuracy. Newer research from 2025 found the best tools did far better on long, unedited passages while others stayed weak. Treat any one number with care and ask what it was measured on.
Is AI detection reliable enough to punish a student?
Not on its own. Studies show false positives happen, especially for second-language writers and short or formal texts. University guidance, including from Jisc in the UK, says decisions should never rest solely on detection. A score can justify a conversation and a closer look at drafts and process. It shouldn't be the evidence that decides the outcome.
Why do accuracy claims from detector companies look so high?
Vendors often test on long, clean, unedited AI text in English, which is the easiest case. Headline accuracy can also hide the false positive rate by blending it with other results. The RAID benchmark researchers noted that many detectors claim 99% accuracy or more, yet struggled under harder, more realistic conditions such as paraphrasing and unfamiliar models.
Can AI detectors detect text from newer AI models?
Sometimes, but performance can drop when a detector meets a model it wasn't trained on. The RAID benchmark found unseen models were one of the main ways detectors were fooled. Detectors that are retrained regularly have a better chance, but no one can promise they'll keep up with every new model the day it's released.
Are AI detectors biased against non-native English speakers?
Research says this is a real risk. In a 2023 Stanford study, seven detectors wrongly flagged an average of 61% of TOEFL essays written by non-native speakers, while essays by US eighth graders were classified almost perfectly. Writers using a second language often choose simpler, more predictable words, which can look like AI text to a detector.
Does a higher AI score mean more of the text was written by AI?
Not exactly. Most scores show how likely the text is to be AI-written, based on its patterns. A score of 60% doesn't mean 60% of the words came from AI. Sentence-level scores give a better picture of where the AI-like patterns sit, but even those are estimates that need a human to check them.
Sources
Testing of detection tools for AI-generated text, Weber-Wulff et al., International Journal for Educational Integrity, December 2023
GenAI Detection Tools, Adversarial Techniques and Implications for Inclusivity in Higher Education, Perkins et al., International Journal of Educational Technology in Higher Education, 2024
GPT detectors are biased against non-native English writers, Liang et al., Patterns, July 2023
RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors, Dugan et al., ACL 2024, May 2024
Artificial Writing and Automated Detection, Jabarian and Imas, NBER Working Paper 34223, September 2025
Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods, Fraser, Dawkins and Kiritchenko, Journal of Artificial Intelligence Research, 2025
AI Detection and assessment: an update for 2025, Jisc National Centre for AI, June 2025
Evaluating the efficacy of AI content detection tools in differentiating between human and AI-generated text, Elkhatat, Elsaid and Almeer, International Journal for Educational Integrity, September 2023
Comments