An AI detector never watches anyone type. It gets a block of text and nothing else, and it has to guess where that text came from. So how do AI detectors work with only words on a page? They measure how predictable those words are, compare the patterns with large sets of human and AI writing, and turn what they find into a probability.
Once you know what a detector measures, you'll know when to take its score seriously and when to set it aside.
Below we walk through the whole process in plain English, from tokens and perplexity to burstiness, trained classifiers, sentence scores and watermarks. Then we cover where detectors struggle, with research to back each point.
How do AI detectors work?
AI detectors work by running text through a language model and measuring how predictable each word is in its context. Text from AI writing tools tends to choose likely words again and again, so it reads as very predictable and very even. Human writing usually has more surprises and more variety. A trained classifier combines these signals and outputs a probability that the text, and each sentence in it, was AI-generated.
What does an AI detector actually read?
It reads tokens. A token is a chunk of text that a language model treats as one unit, often a whole word, sometimes part of one or a comma. "Detectors" might be split into "detect" and "ors".
Large language models write one token at a time. At each step, the model looks at everything written so far and gives every possible next token a probability. Then it picks one. AI chatbots usually favor the likely picks, which keeps their writing fluent and on topic.
A detector runs that process in reverse. It doesn't write anything. It reads your text token by token and asks, at each step, how likely a model would have been to choose that exact token. That's the core signal behind most AI detection in writing.
This idea is older than today's chatbots. In 2019, Sebastian Gehrmann, Hendrik Strobelt and Alexander Rush published a research demo that colored each word by how highly a language model ranked it. In their tests, the color coding raised people's ability to spot generated text from 54% to 72%, with no training at all.
What is perplexity, in plain English?
Perplexity is a measure of how surprised a language model is by a piece of text. Low perplexity means the model saw almost every word coming. High perplexity means the text kept doing things the model didn't expect.
Technically, perplexity is the exponentiated average negative log-likelihood of a sequence, as the Hugging Face documentation puts it. In plain terms, you take how surprised the model was at each token, average it across the text, and turn that average back into a single number.
A worked example
Say a model reads "I take my coffee with milk and" and has to guess the next word. Here's how it might spread its bets. The numbers below are made up for illustration, not taken from a real model.
Next word | Illustrative probability | Surprise level |
|---|---|---|
sugar | 60% | Very low |
honey | 5% | Moderate |
cinnamon | 2% | High |
regret | under 0.1% | Very high |
Now compare two endings.
I take my coffee with milk and sugar.
I take my coffee with milk and regret.
The first sentence has low perplexity. Every word is close to the model's top guess. The second ends on a word the model gave almost no chance, so its perplexity is higher.
Neither sentence was written by AI. That's the catch with perplexity. It measures predictability, and people write predictable sentences all the time, especially in lab reports, legal writing and school essays.
What is burstiness?
Burstiness is how much a text varies from one sentence to the next. It covers sentence length, sentence structure and how predictability rises and falls across the text.
People tend to write in bursts. A long, winding sentence, then a short one. An odd word choice, then a plain stretch. Text from AI writing tools is often more uniform, with sentences of similar length that each make one tidy point.
Here's an example paragraph written to show those even patterns.
Now here's a version with more burstiness.
I hated running until I got a dog. She needs two walks a day, rain or not, and somewhere in the second month my resting heart rate dropped. Sleep got better too. I didn't plan any of it.
The second version has a specific person, uneven rhythm and details a model wouldn't guess. But a person could easily write the first one too, and that's where false positives come from. The Gonzaga University faculty guide on AI detectors makes the same point. Formal academic writing pushes perplexity and burstiness down, so it can look like AI output.
How does a trained classifier decide?
A classifier is a model trained on a large set of labelled examples, some human-written and some AI-written, so it learns which features separate the two. Most modern detectors are classifiers. They don't rely on one number like perplexity.
The features a classifier learns can include perplexity and burstiness, plus word choice, sentence shape, punctuation habits and many patterns no person would think to name. A 2025 review by Kathleen Fraser, Hillary Dawkins and Svetlana Kiritchenko at the National Research Council Canada found that the most effective detectors are trained on data from the same domain, language and model settings as the text they're checking. A detector trained mostly on news articles will do worse on poetry.
A second family, often called zero-shot detectors, skips the training and uses a language model's probabilities directly. In 2023, Eric Mitchell and colleagues at Stanford made small random rewrites of passages and watched the probability. AI-written text tends to sit at a local peak, so almost any small change makes it less likely. Human text shows that pattern less.
Signal | What it measures | Why it can point to AI | Where it misleads |
|---|---|---|---|
Token probability | How likely each word was in context | AI tools favor likely words | Plain, simple human writing |
Perplexity | Average surprise across the text | AI text is often low-surprise | Formal and formulaic writing |
Burstiness | Variation between sentences | AI text is often even | Short or tightly edited texts |
Learned features | Patterns from labelled examples | Picks up subtle habits | Topics or languages it wasn't trained on |
Watermark | A hidden statistical pattern | Proves a specific tool wrote it | Only works if the tool added one |
How do detectors score each sentence?
Sentence-level scoring splits the text into sentences and gives each one its own probability, using the surrounding text as context. The overall score then combines those sentence scores.
So a document can get a middling overall score with a few sentences highlighted. Maybe a student pasted in one polished paragraph. Maybe they wrote all of it and that paragraph just happens to be predictable. The sentence view shows you where to look, without telling you which story is true.
Here's the process most detectors follow, simplified.
Clean the text and split it into tokens and sentences.
Run the tokens through a language model to get a probability for each one.
Turn those probabilities, plus other features, into a score for each sentence.
Combine the sentence scores into one overall probability.
Report the result, ideally with a range and a confidence level so you can see how sure the model is.
WrittenCheck follows this shape. It gives an overall score, a likely range, a confidence level and a score for every sentence, and it highlights the sentences that read most like AI. You can read more on our how it works page.
Why do AI detectors give a probability instead of a yes or no?
AI detectors give probabilities because human and AI writing overlap. Plenty of human text is predictable, and some AI text is surprising. No single cut-off separates them cleanly, so the honest output is a likelihood.
Someone still has to pick where "likely AI" starts, and that choice matters a lot. In the RAID benchmark, presented at ACL 2024, Liam Dugan and colleagues at the University of Pennsylvania argued that detector accuracy swings so much with this threshold that it must always be reported. With some simple default thresholds, a few metric-based detectors flagged almost every human text as AI. So the researchers set each detector's threshold so that only 5% of human texts would be wrongly flagged, then compared them.
So what does a score of 70% mean? It means the text looks more like the AI-written examples the detector learned from than the human ones. It doesn't mean 70% of the words came from AI. It doesn't mean there's a 70% chance someone broke a rule.
What is AI watermarking, and do detectors use it?
AI watermarking is a hidden statistical pattern that an AI tool adds to its own output as it writes. In a 2023 paper, John Kirchenbauer and colleagues at the University of Maryland described a simple version. Before each token, the tool secretly marks part of its vocabulary as a "green list" and nudges itself toward those words. A reader can't see it. Someone with the key can count green words and check whether there are far more than chance would allow.
Watermarks only help if the AI tool added one and the checker has the key. Most pasted text carries no watermark anyone can check, and watermarks can be worn down. Vinu Sankar Sadasivan and colleagues at the University of Maryland showed in 2023 that running text through a paraphrasing tool again and again can sharply cut detection rates, for watermarks and for other detectors, with only a small drop in quality. So most AI detection still depends on the statistics described above.
Where do AI detectors struggle?
AI detectors struggle wherever human and AI writing look alike, or where there's too little text to judge. Research points to five weak spots.
Short texts. Fewer words means fewer clues. The 2025 Fraser review reports that classifiers need roughly 120 to 200 words to reach their full potential. WrittenCheck asks for at least 80 words for the same reason, and longer is better.
AI drafts edited by people. In a 2023 study in the International Journal for Educational Integrity, Debora Weber-Wulff and colleagues tested 14 tools. Accuracy on unedited AI text averaged 74%. On AI text that people had edited by hand, it fell to 42%. On machine-paraphrased AI text, it fell to 26%.
Second-language writers. In 2023, James Zou's team at Stanford ran 91 essays written by non-native English speakers for the TOEFL exam through seven detectors. On average, 61% were wrongly flagged as AI. Essays by US eighth graders were classified almost perfectly. The researchers linked this to lower perplexity, since writers working in a second language often use a smaller range of words.
Formal and well-known texts. Writing that follows strict conventions is predictable by design. Famous texts are even more predictable, because language models were trained on them. The Gonzaga guide notes that detectors have flagged parts of the Bible and the US Constitution as AI-generated.
New AI models and unusual settings. The RAID benchmark found detectors were easily fooled by AI models they hadn't seen in training, by changes in how text was sampled and by deliberate attacks.
How should you use an AI detector's result?
Use it as one input, and make sure it's a fair one. Jisc, the UK's education technology body, put it plainly in its 2025 update on AI detection. Decisions "should never be based solely on AI detection, whether by a tool or a person."
Check the length. Under about 200 words, give the score little weight.
Read the highlighted sentences. Do they share a style, or are they spread out at random?
Look at the range and confidence. A wide range or low confidence means the detector isn't sure.
Consider the writer. Second-language writers and formal genres raise the odds of a false positive.
Look for process evidence. Drafts, notes and version history in Google Docs or Microsoft Word say far more than a score.
Talk to the writer. Ask them to explain an argument or a word choice. People can usually talk through work they wrote.
To see these signals on your own writing, paste a draft into our free AI detector. The sentence scores show which lines read most like AI.
Frequently asked questions
What is an AI detector?
An AI detector is a tool that estimates how likely a piece of text is to have been written by an AI writing tool. It doesn't search a database of AI output. It reads the text with a language model, measures patterns like predictability and variation, and gives a probability. Good detectors also show which sentences drove the result, so you can check them yourself.
How do AI detectors work, in one sentence?
They read your text with a language model, measure how predictable each word and sentence is, and compare that pattern with what they learned from labelled human and AI writing. The result is a probability, so it's an estimate about the text and never proof about who wrote it.
How is AI detected in writing if there's no hidden mark?
Most AI detection relies on statistics. Language models tend to choose likely words, so their writing is more predictable and more even than most human writing. A detector measures that predictability at the level of each word and sentence, then compares the pattern with what it learned from labelled human and AI examples. Watermarks exist but are rarely available to check.
Is low perplexity proof that AI wrote something?
No. Low perplexity only means the words were easy for a model to predict. Lab reports, legal documents, simple instructions and essays by people writing in a second language can all be very predictable. That's why perplexity alone is a weak test, and most detectors combine it with many other features.
Why did my own writing get a high AI score?
Your writing may simply be predictable to a language model. Short texts, formal or formulaic styles, heavy use of grammar tools and writing in a second language all push scores up. Look at which sentences were highlighted. If they're the most generic ones, adding your own examples and varying your sentences is good writing advice anyway, with or without a detector.
Do AI detectors keep or reuse the text I paste in?
It depends on the tool, so read its privacy policy. WrittenCheck runs on our own server, doesn't send your text to any other AI service and doesn't keep it. If you upload a file, it's read in your browser and only the text is sent for checking.
Sources
GLTR: Statistical Detection and Visualization of Generated Text, Gehrmann, Strobelt and Rush, ACL 2019 demo track, June 2019
Perplexity of fixed-length models, Hugging Face Transformers documentation, accessed October 2026
AI Detectors (A Guide to AI for Gonzaga Faculty), Gonzaga University Library, updated May 2026
Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods, Fraser, Dawkins and Kiritchenko, Journal of Artificial Intelligence Research, 2025
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature, Mitchell et al., ICML 2023, January 2023
RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors, Dugan et al., ACL 2024, May 2024
A Watermark for Large Language Models, Kirchenbauer et al., ICML 2023, January 2023
Can AI-Generated Text be Reliably Detected?, Sadasivan et al., Transactions on Machine Learning Research, March 2023
Testing of detection tools for AI-generated text, Weber-Wulff et al., International Journal for Educational Integrity, December 2023
GPT detectors are biased against non-native English writers, Liang et al., Patterns, July 2023
AI Detection and assessment: an update for 2025, Jisc National Centre for AI, June 2025
Comments