Detect AI-written text: what AI detectors get wrong

Blog · Detect AI-written text

Can you detect AI-written text just by reading it? Usually not. In a large study, readers without practice did no better than chance. People who write a lot with ChatGPT themselves, on the other hand, get it right almost every time, because they know where to look. AI detectors rely on similar signals (word choice, sentence structure), but all they return is a probability, and they keep flagging plain, clear human writing as machine-made.

In brief

Untrained readers spot AI text about as well as a coin toss; frequent ChatGPT users are almost always right.
A detector measures how a text is built. It cannot prove who wrote it.
Google asks whether content helps, not how it was produced.

Published 26 Sep 2026Nikolai Schöbel and Jeremias Burger8 min read

Nikolai Schöbel and Jeremias Burger

Co-founders of Scalableloops GmbH. Nikolai Schöbel leads online marketing and AI strategy, Jeremias Burger the AI architecture. Both build AI systems and train teams on them in their own agency work.

On this page

Can AI-written text be detected at all?
What gives an AI text away to trained readers?
What does an AI detector actually measure?
How often are AI detectors wrong?
What does a count of German texts show?
Does AI-written content hurt your Google ranking?
Do you have to label AI-generated text in the EU?
How do you write text that sounds like you?
Frequently asked questions
How to handle AI-written text in your company
Where the information on this page comes from

Basics

Can AI-written text be detected at all?

Yes, just not the way most people think. In a 2023 study published in PNAS, researchers showed around 4,600 participants short self-descriptions of the kind you find in professional profiles, holiday rental listings or dating profiles, and asked: human or machine? Participants were right 50 to 52 percent of the time. A coin would have done just as well.

What we find more interesting than the rate is what people relied on. A spelling mistake, a sentence starting with “I”, a mention of family, and a text was judged human, even though these features say almost nothing about where a text came from.

The picture changes completely when the judges write with ChatGPT a lot themselves. In a University of Maryland study, five such heavy users assessed 300 non-fiction articles, and their majority vote classified 299 of them correctly, including texts that had been paraphrased afterwards (presented in 2025 at ACL, the leading computational linguistics conference). Practice beats gut feeling, and anyone who reviews texts for a living can learn it.

Signals

What gives an AI text away to trained readers?

Word choice, most often. Language models have favourite words; in English the classic is “delve”, which turned up so often in scientific abstracts after 2023 that researchers used it to estimate how much of an entire year's output was written with AI help: at least 13.5 percent of the 2024 abstracts, with “delves” appearing 28 times more often than expected.

The second clue is sentence structure, and in AI text it is remarkably well-behaved. Sentences run to roughly the same length, there are lots of nouns, and nothing snags. A 2025 analysis in PNAS counted this across thousands of texts and found chat models using far more nominalisations than people. Oddly enough, the raw base versions of the same models, before they were trained to give friendly chat answers, wrote more like humans.

What we notice most when reading, though, is the missing stance. People write “I think” or “probably” when they are not sure, and in a comparison of argumentative essays they did so more often than ChatGPT, which prefers smooth statements.

Signal	Typical of AI text	Typical of human text
Word choice	Recurring pet words, polished filler	Plain, sometimes unusual, very specific words
Sentence length	Even, mostly medium	Varies a lot, the occasional very long sentence
Structure	Lists of three, “not only … but also”	Irregular, with asides and brackets
Stance	Smooth statements	Hedges such as “probably” or “I think”
Repetition	Swaps synonyms but repeats ideas	Repeats simple words, rarely whole ideas

Technology

What does an AI detector actually measure?

At its core sits a surprisingly simple idea. A language model picks words that another language model can predict well, whereas people reach for a word nobody saw coming more often. How surprising a text is to a model is called perplexity. And for how much that varies from sentence to sentence, the detector company GPTZero popularised the term burstiness.

Newer tools compare two models with each other, or learn from large collections of known human and machine texts how the two differ. In the end there is always a number, usually shown as a percentage. That number describes the statistics of the text, not who wrote it, and the difference is easy to overlook.

Reliability

How often are AI detectors wrong?

More often than the percentage suggests. Even OpenAI, the company behind ChatGPT, took its own detector offline again in July 2023 because it was not accurate enough. By OpenAI's own published figures, it correctly flagged only 26 percent of AI texts and wrongly labelled 9 percent of human texts as AI-written.

It gets really uncomfortable for people who write plainly and clearly. In 2023 a research team ran English essays through seven common detectors: essays written for a language test by non-native speakers were classified as AI 61 percent of the time on average, while essays by American eighth graders were sorted almost perfectly. Why? A simple vocabulary is predictable, and predictability is exactly what detectors react to.

So a single detector score is a hint, nothing more. Anyone who uses it to judge a job application, a term paper or a supplier they want to accuse may be treating unfairly someone who simply writes cleanly.

Our own measurement

What does a count of German texts show?

Almost all studies on the subject work with English, so we counted for German ourselves. On one side were 155 newspaper and trade articles written by people, on the other 65 texts from a current language model (Claude) on similar topics.

We did not expect the clearest result, because it contradicts a popular writing rule. Short sentences are not a sign of a human; long ones are. In the human texts, 15.6 percent of sentences had 25 words or more, in the model's texts only 2.8 percent, and the model wrote twice as many very short sentences. Brackets like these (for a thought that occurs to you along the way) appeared regularly in human writing and not once in the typical model text, and words like “probably” behaved the same way.

Of course 220 texts are not a large study, and a count like this says nothing about which text is better. It does show one thing you will not find in most style guides: if you build every paragraph from short, punchy sentences, you now sound more like a machine than a person.

Search

Does AI-written content hurt your Google ranking?

No, at least not because an AI wrote it. Back in February 2023, Google stated that its search rewards helpful, reliable content however it is produced. What Google does not like are pages produced in bulk mainly to show up in search results while offering readers little; since March 2024 that counts as spam, whether a person or a machine wrote it.

So the old questions still decide. Is what it says true? Does someone arriving with a specific question find the answer here, or just sentences they have read many times before? And can you tell that someone who knows the subject stands behind it? How Google assesses this is covered in our article on E-E-A-T.

Regulation

Do you have to label AI-generated text in the EU?

It depends on who you are and what the text is for. Article 50 of the EU AI Act first obliges the providers of AI systems to mark their output in a machine-readable way, for new systems since 2 August 2026, and for systems already on the market before that from 2 December 2026. Anyone who publishes texts informing the public on matters of public interest must disclose that they were generated by AI, unless a person has reviewed them editorially and takes responsibility for them.

For typical business content, product pages or a newsletter that someone in your company reads and approves anyway, this is unlikely to trigger a labelling duty. Chatbots that talk directly to your customers are a different matter; what applies there is explained in our article on AI disclosure for chatbots, and how providers mark their texts technically is covered in the article on watermarks in AI text.

In practice

How do you write text that sounds like you?

Not with tricks meant to fool a detector. They change nothing about whether a text is any good, and anyone who suddenly finds slang in a product description senses the intent. What helps is old-fashioned craft: follow a thought through even if the sentence gets longer, say where you are not sure, and prefer the precise word over the general one (instead of “an important feature”, say what the feature saves your customer).

At our company every text goes through two checks before it is published: one on facts, with two independent sources for each claim, and one on style. We use the detector only as a warning light, and when it goes off, it is usually because of paragraphs that neatly summarise what was already said. We cut those, and the text gets shorter and better.

Frequently asked questions

Can you prove a text was written by ChatGPT?

No, not with a detector. It calculates a probability and can be wrong in both directions. Invisible watermarks that some providers build into their output come closer to proof, but they also disappear once someone rewrites the text thoroughly.

Are free AI detectors accurate?

Nobody knows exactly, because hardly any provider discloses how its tool works. What is known is the weak spot: with plainly written texts and with people writing in a language that is not their first, detectors raise false alarms remarkably often.

Does Google know if content was written by AI?

For rankings, Google says it does not matter. In 2023 the company stated that it rewards good content however it is produced. Trouble comes to pages mass-produced to occupy search results, whether a person or a machine wrote them.

How can I make AI-written text sound more human?

Treat it like a draft from a new colleague. Check the claims, cut the filler, add your own experience, and let a long sentence stand now and then. Deliberate typos do not help; they measurably cost readers' trust.

Can I use AI-written text on my website?

Yes. But you are responsible for what it says, just as with anything you wrote yourself. If the topic is of public interest, it is worth checking Article 50 of the EU AI Act.

Can teachers and reviewers spot AI text?

That depends on how much they write with AI themselves. The heavy users in the 2025 study were almost always right; readers without that experience were barely better than a coin toss in other research.

Next steps

How to handle AI-written text in your company

01
Set up sign-off
Decide who reads AI drafts and who is accountable for them being correct.
02
Check facts twice
Every figure and every name needs a source that someone on your team has actually read.
03
Use detectors as a warning light
If a detector flags a text, take a second look at the sentences. Do not use it to judge people.
04
Clarify labelling
For chatbots and texts on public-interest topics, check whether Article 50 of the EU AI Act applies.