What Is an AI Detector? How They Work, and Why They're Wrong So Often

An AI detector estimates whether text was machine-written using perplexity and burstiness. Accuracy claims vs the 61% false-positive study, explained.

An AI detector is a tool that reads a piece of text and estimates the probability that it was written by AI. Millions of people search for one every month — teachers checking essays, editors checking freelancers, recruiters checking cover letters. Here’s the part the marketing pages don’t lead with: the most consequential thing about AI detectors in 2026 is how often their guesses are wrong, and what those wrong guesses cost the humans on the receiving end.

TL;DR. An AI detector analyzes text statistics — mainly perplexity (predictability) and burstiness (sentence variation) — to estimate whether writing is machine-generated. It outputs a probability, not proof. Stanford HAI research (2023) found detectors falsely flagged 61.3% of non-native English speakers’ essays; universities including Washington State (2026) and UC-Berkeley no longer accept detector scores as evidence.

Last reviewed: 2026-07-13

What is an AI detector?

An AI detector (also called an AI checker or AI-content detector) is software that takes a passage of text and returns an estimate — usually a percentage — of how likely that text is to have been produced by a large language model like ChatGPT, Claude, or Gemini. Well-known examples include Turnitin’s AI-detection module, GPTZero, Originality.ai, and Copyleaks. The critical word in that definition is estimate: no detector can see who typed the words, so every score is a statistical guess about writing style, not a record of what happened.

That distinction gets lost the moment a score appears on a screen. “83% AI” reads like a measurement, the way a thermometer reads temperature. It isn’t one. It means: this text’s statistical patterns resemble patterns common in AI-generated text. Human writing can match those patterns for entirely human reasons — which is where the trouble starts, and why understanding the mechanism matters before trusting the number.

How do AI detectors work?

AI detectors work by measuring statistical properties of text that tend to differ between human and machine writing, then scoring how “AI-like” those properties look. The two measurements nearly every detector relies on are perplexity and burstiness — and understanding them explains both why detectors sometimes work and why they systematically fail certain writers.

Perplexity measures how predictable each next word is. Language models write by predicting probable next words, so AI text tends to be smooth and statistically unsurprising. If a passage keeps choosing exactly the words a model would predict, its perplexity is low, and a detector reads that as machine-like.

Burstiness measures variation in sentence structure and length. Human writers naturally mix short punchy sentences with long winding ones; AI models (especially older ones) tended to produce more uniform rhythm. Low variation reads as machine-like.

What happens when you paste text into a detector
Your text goes in essay, cover letter, article — the tool sees only the words
Statistical scan perplexity (predictability) + burstiness (sentence variation)
Pattern comparison how closely do these stats resemble known AI output?
A probability comes out "83% AI" — a resemblance score, not a witness
A statistical comparison — at no point does the tool learn who actually wrote the words

Notice what’s not in that pipeline: any knowledge of the writing process, the writer, drafts, or intent. Stanford HAI’s analysis (2023) identified this surface-statistics approach as precisely why detectors misfire on real humans — anyone whose writing is simple, formulaic, or vocabulary-constrained produces low perplexity naturally. That describes non-native English speakers, younger students, technical writers following templates, and anyone writing under rigid instructions.

There’s also an arms-race problem. The WSU Provost’s review (2026) notes that “humanizer” tools that lightly rewrite AI output readily defeat current detectors — so the genuinely dishonest text slips through while honest-but-plain writing gets flagged. Small edits swing scores dramatically in both directions, which is not a property you want in evidence.

Are AI detectors accurate?

The honest answer: accurate enough to be interesting at the batch level, and unreliable enough that using a single score to accuse a single person is indefensible — which is the exact use case most people have in mind. The evidence has piled up from three directions: independent research, vendor admissions, and institutions abandoning the tools.

EvidenceFindingSource
Stanford study (Liang et al.)61.3% of TOEFL essays by non-native English speakers falsely flagged; 18 of 91 flagged by all seven detectors tested; native-speaker essays under 10%Stanford HAI / Patterns (2023)
OpenAI’s own detectorShut down after reaching roughly 26% accuracy — the maker of ChatGPT couldn’t reliably detect ChatGPTOpenAI (2023)
Turnitin’s claim vs independent reviewVendor claims ~1–2% false positives; WSU’s review notes independent studies place it higher, especially for neurodivergent and ESL writersWSU Provost memo (2026)
WSU case data33% of detector-based academic-integrity cases (2023–2025) ended “not responsible”WSU Provost memo (2026)
Institutional retreatWSU cancelled Turnitin AI detection in February 2026, joining UC-Berkeley, Indiana, Michigan State, Oregon State, and the University of Washington; the University of the Free State followed in July 2026WSU Faculty Senate (2026); UFS (2026)

The Stanford HAI team (2023) showed the bias mechanism is structural, not a bug to patch: detectors reward “linguistic diversity,” so writers with constrained vocabulary — by language background, age, or disability — get systematically misread as machines. The WSU Provost memo (2026) adds that even trained humans who use AI daily misjudge authorship at least 4% of the time, worse than Turnitin’s claimed rate — nobody, human or machine, reliably “spots AI” from style alone.

OpenAI’s own education lead, per Fortune’s reporting (2026), described early detector deployment in schools as getting off on the “wrong foot,” with wrongful accusations of students who hadn’t cheated. When the vendor, the researchers, and the universities converge on the same conclusion, the question stops being “which detector is best” and becomes “what do we do instead.”

Why a probability can’t be proof

Even a detector with a true 98% accuracy rate produces a flood of false accusations at scale — this is the base-rate problem, and it’s arithmetic, not opinion. Run 1,000 honest essays through a 2%-false-positive detector and you get 20 wrongly flagged students. A large high school does that volume in a couple of weeks. The WSU case review (2026) found a third of AI-related integrity hearings collapsing for exactly this reason.

The human cost is concrete. Bloomberg’s reporting (2024) documented student Moira Olmsted being falsely accused after a detector flagged writing she had produced herself — one of the cases that now circulate in educator threads with thousands of shares every exam season. False flags cluster on students least equipped to fight them: multilingual, neurodivergent, first-generation. That’s why the emerging institutional standard treats a detector score as, at most, a reason to look closer — never as the evidence itself.

What this means for teachers

The productive question isn’t “which detector should I buy” — it’s “what’s my classroom AI rule, and what’s my process when I suspect misuse.” The evidence-based answer is process artifacts (drafts, revision history), a short oral check (“walk me through this paragraph”), and assignments AI can’t complete alone. Our guides on the 10-minute classroom AI rule and AI-resistant assignment redesign cover the exact workflow, the detector false-positives deep dive has the full evidence file plus what to tell an administrator pushing a scanner, and the Teaching with AI course packages the whole toolkit.

What this means for professors and college instructors

Your institution may already prohibit detector-only accusations — Washington State’s policy (2026) explicitly bars AI detection from integrity cases, and its memo lists UC-Berkeley, Indiana, Michigan State, Oregon State, and the University of Washington alongside it — so check before you screenshot a score into an email. The university teaching-center pattern (Stanford’s CTL among them) is a syllabus AI statement per course plus process evidence when questions arise. A structured academic-writing workflow with disclosure beats surveillance, and it’s defensible in a grade appeal, which the score is not.

What this means for recruiters and HR

AI-polished cover letters and résumés are now the norm, not the exception — running applicants through a detector mostly measures who used a template, and rejecting on a score invites exactly the demographic bias documented in the Stanford findings (2023), concentrated on the non-native English speakers in your pipeline. Assess with structured interviews and work samples; if writing quality matters for the role, test it live. Our résumé-writing course shows what candidates are actually doing with AI, which is useful intelligence for reading applications fairly.

What this means for editors and content managers

Detector-screening freelancers produces false accusations against your plainest-style writers and misses anyone who paraphrases — the double failure the WSU review (2026) describes. The workable standard is a disclosure clause in the contract plus quality-and-fact review of the work itself. If a piece is wrong, generic, or invented, you don’t need a detector to reject it — and if it’s accurate and good, the score wasn’t the point. A clear-writing standard in your style guide does more for quality than any scanner.

What this means for freelancers and students who get flagged

Don’t panic, and don’t confess to something you didn’t do. Bring your process: drafts, version history, notes, the ability to explain your own paragraphs out loud — the Study Smarter with AI course covers building exactly that habit. Ask what evidence exists besides the score. The institutions that have studied this, including the WSU Provost’s office (2026), explicitly instruct that a score alone can’t carry an accusation — and the Stanford figures (2023) are your citation if you’re a non-native English writer facing one.

Detection vs. watermarking: where this is heading

The detector approach — guessing after the fact — is slowly being displaced by approaches that mark AI content at creation. Google’s SynthID embeds an invisible statistical watermark in AI-generated text, images, and audio as they’re made; the C2PA standard attaches a cryptographic provenance trail to media. Google DeepMind’s documentation (2024) shows watermark detection of watermarked content is dramatically more reliable than statistical guessing at unmarked text.

The catch: watermarking only identifies content from tools that chose to watermark, and text watermarks weaken under heavy editing. So provenance solves the “did this come from our AI” problem for cooperating platforms — it doesn’t resolve the classroom question of what a student did last night. That one gets solved by assignment design and process evidence, not by any scanner, which is why the profession-level advice above doesn’t expire when the technology improves.

Common misconceptions

Four claims about AI detectors circulate constantly — in staff rooms, in HR meetings, in the marketing for the detectors themselves — and each one collapses against the published evidence. They’re worth addressing directly because every one of them leads someone to trust a score more than the score deserves.

“A high score means the person used AI.” A high score means the text’s statistics resemble AI output. Plain, careful, non-native, or template-following human writing produces the same statistics. That’s not an edge case — it’s a 61%-of-a-Stanford-sample case.

“Detectors are improving fast, so the problem is temporary.” The failure is structural: detectors read style, style overlaps between humans and machines, and the overlap grows as models improve and as humans write more like models. OpenAI abandoned its detector rather than iterate on it.

“A 1–2% false-positive rate is basically fine.” At classroom scale, 1–2% is multiple wrongly accused students per teacher per year, concentrated on vulnerable writers — and the WSU data (2026) shows a third of detector-based cases failing on review.

“If detectors don’t work, cheating is unstoppable.” Detection was never the effective control. Clear rules with disclosure, process-based assignments, and short oral checks are — they’re older than AI, and they don’t generate false accusations.

Frequently asked questions

How do AI detectors work?

AI detectors measure statistical properties of text — mainly perplexity (how predictable each next word is) and burstiness (how much sentence structure varies) — and compare those patterns against known AI-generated writing. They output a probability that the text is machine-written. They have no access to the writing process, so the score is a resemblance estimate, never a record of authorship.

Are AI detectors accurate enough to accuse someone?

No. Stanford research found detectors falsely flagged 61.3% of essays by non-native English speakers, OpenAI shut down its own detector at roughly 26% accuracy, and Washington State University found a third of detector-based integrity cases ended “not responsible.” Universities including UC-Berkeley and the University of Washington have banned detector scores as primary evidence of misconduct.

Why do AI detectors flag human writing as AI?

Because they read surface statistics, not authorship. Writers with simpler vocabulary or uniform sentence rhythm — non-native English speakers, younger students, neurodivergent writers, anyone following a strict template — naturally produce the low-perplexity, low-burstiness patterns detectors associate with machines.

What should teachers use instead of an AI detector?

A clear classroom AI rule with a disclosure requirement, assignments that include process artifacts (proposals, drafts, reflections), a short oral component tied to submitted work, and a conversation-first procedure when something seems off. These catch misuse more reliably than scanners and don’t produce false accusations.

Can AI detectors be beaten by “humanizer” tools?

Yes, easily — and that asymmetry is the core problem. Paraphrasing tools readily rewrite AI text to pass detectors, while honest plain writing keeps getting flagged. A control that misses determined cheaters and catches innocent writers fails in both directions at once.

The bottom line

An AI detector is a probability generator, and probabilities can’t carry accusations. The evidence from Stanford (2023), from the universities that quit (2026), and from the vendor that shut its own detector down all points the same way: the durable answers are rules, process, and assignment design — human moves, all learnable, and exactly what the courses above teach.

See also

Everything below connects to the same underlying skill — using AI honestly and judging AI use fairly — organized by what you’re trying to do next: learn the workflow in a structured course, look up an adjacent term, read the practical guides, or grab a ready-made prompt template.

Courses: Teaching with AI · AI for Teachers: End-of-Year 5-Prompt Routine · Academic Writing with AI · Better Writing with AI · Resume Writing with AI · Study Smarter with AI · AI for Scientific Writing · Professional Email Writing · Storytelling & Creative Writing with AI · Grant Writing with AI

Related terms: SynthID · C2PA · AI hallucination · Frontier model · Context window · Agentic AI · Answer engine optimization

Guides: AI detector false positives: what teachers should do instead · The 10-minute classroom AI rule + parent email · Rewrite any assignment so ChatGPT can’t do it · Best AI tools for teachers · Teachers: set up next year with ChatGPT · ChatGPT for teachers: end-of-year routine · College consultants: the AI line in the sand

Skills: Essay Grader · Academic Writing · Writing Clearly · Essay Argument Builder · College Application Essay Coach · Writing Style Matcher

Sources

Build Real AI Skills

Step-by-step courses with quizzes and certificates for your resume