Why Fake Customers Make AI Worse at Predicting Real Ones

Marketers have started using AI in a clever-sounding way. Before spending money to launch an ad or a headline, they ask a large language model — a system like ChatGPT that predicts text — to pretend to be their target customer. "You are a 34-year-old suburban mother of two who reads news on her phone. Would you click this headline?" Ask ten such invented characters, count the votes, and you get a cheap forecast of how real people will react.

These invented characters are called synthetic personas: AI instructed to role-play a specific demographic profile so its answers stand in for a human being of that type.

The appeal is obvious. Real audience testing is slow and expensive. A panel of fake customers is instant and free. And earlier research seemed to bless the idea — profile-conditioned language models can reproduce the answer patterns of real survey samples fairly well.

A new study put the method to a hard test. The result is not "personas are a bit weak." It is stronger and stranger than that: the personas made the AI worse. A plainer version of the same AI, given none of the persona instructions, predicted real human behavior markedly better.

The test they built

To check whether a prediction is valid, you need real behavior to check it against. The researchers used the Upworthy Research Archive — thousands of genuine headline A/B tests. In an A/B test, two versions of the same article headline are shown to real visitors split at random, and you measure which one gets more clicks. This is ground truth: not what people say they'd do, but what they actually did.

They then set up a contest between two ways of using the AI.

• The persona panel. Ten synthetic personas, built to match the real audience's actual demographics. Each was asked which headline it would click.
• The no-persona baseline. The same model, asked one flat question: "How likely is a typical reader to click this?" No character, no role-play, no profile.

Both predicted a winner for each test. The winners were then compared to what real traffic chose.

The catch that almost nobody checks

Before the headline result, there is a quieter lesson worth carrying away.

Most of the A/B tests had no real winner. The click-rate difference between the two headlines was too small to be distinguishable from random noise. If you "validate" a prediction against a coin flip, you learn nothing — you're grading the AI against results that were never real to begin with.

So the researchers threw out the unreliable tests and kept only the ones with a statistically solid winner: 399 of them. This is the binding constraint they name explicitly — your evaluation can never be better than the reliability of the thing you're measuring against. Skip this step and you can "prove" almost anything, because you're measuring against noise that will happily agree with whatever you claim.

That principle travels far beyond marketing. Backtesting a trading strategy against price moves that are mostly noise. Judging a medical test against diagnoses that were themselves uncertain. Ranking schools on score differences smaller than the year-to-year wobble. In each case, a wrong answer isn't the danger — a meaningless answer dressed as a real one is.

The result

On the reliable 399, the two methods diverged sharply.

The scores measure how well each method's ranking of headlines matched the real ranking. On the standard measure, the no-persona baseline hit a medium-strength correlation and picked the true winning headline 49% of the time. The persona panel barely beat chance and picked the winner only 35% of the time. The gap was large enough that it clearly wasn't a fluke.

Then they tried to break the result. Different random seeds. Different phrasings of the prompt. Three different sizes of Google's Gemini model. A completely separate model family, OpenAI's GPT-4.1. A different dataset in a different domain. The finding held every time. Personas lost.

Why role-play backfires

Here is the mechanism, and it's the part worth keeping.

When you ask a language model "how likely is a typical reader to click this?", you are pulling on something the model is genuinely good at: a population-level prior. The model has absorbed billions of examples of what people actually click, share, and ignore. Averaged over all of humanity, it has a decent sense of what pulls attention. The flat question lets that broad average speak.

When you force the model to be a specific 34-year-old suburban mother, you do two damaging things at once.

First, you inject bias. The model doesn't have a real 34-year-old suburban mother inside it. It has a stereotype of one — an exaggerated caricature assembled from how such people are described in text, not how they behave. It role-plays the cliché, which is systematically off from the real average.

Second, you inject noise. Ten personas give ten idiosyncratic performances, each drifting in a different direction. You're not sampling ten real people whose errors cancel out. You're sampling ten different theatrical guesses, and their scatter is added on top of the bias.

So personas don't refine the good population-level answer. They corrupt it — first bending it toward a stereotype, then blurring it with random noise. The plain question already contained the wisdom; the costume covered it up.

The portable idea

The trap has a name worth remembering: specificity is not accuracy. A detailed, textured, plausible-sounding answer feels more trustworthy than a bland one. The persona output reads like real market research — quotes, personalities, reasons. That richness is exactly what makes it dangerous, because none of it is grounded in a real person. It's confident fiction.

You will meet this everywhere once you look. The elaborate financial model whose precision comes from assumptions, not data. The vivid customer avatar in a strategy deck. The forecast that gains credibility from its granularity while losing touch with reality. When a model is uncertain, the honest move is often to ask it the broad question and trust the average — not to dress the uncertainty in specific-sounding detail that only feels more real.

Before you trust a confident, detailed prediction, ask the two questions this study forces: Is the thing I'm checking it against actually real, or is it noise? And does the elaborate machinery add signal — or just persuasive-sounding noise on top of a simpler answer that was already better?

Distilled from arXiv cs.AI