The Mood of the Person Grading Your AI

Somewhere behind every chatbot's polite, careful answers sits a person who once clicked "Response A is better than Response B" thousands of times in a row.

That person had a day. Maybe a bad one.

This is the quiet idea at the center of a new paper: preference labels, the bedrock of how models like ChatGPT get tuned to be "helpful," aren't just judgments about which answer is better. They're also a record of the rater's state while judging. Tired, stressed, numbed by hour six of reading distressing content — all of that can leak into the labels.

The authors give this a name: rater state shift. It's different from ordinary disagreement or noisy labeling. It's systematic. If a group of annotators are working under similar stressful conditions, their preferences can drift in the same direction at the same time — and that shared drift can survive averaging, survive aggregation, and quietly bake itself into the reward model. The AI ends up partly trained on how its trainers were feeling.

The paper doesn't claim this happened to any specific model. Instead, it builds a toolkit for finding out: a definition of "correlated rater state bias," a measurable signature called survival-level emotional authenticity (patterns in word choice, tone, and safety language that show up under duress), and five falsifiable predictions with actual effect-size thresholds — the kind of thing you could test on public instruction-tuned models today.

It's a small, sharp reminder: behind every dataset is a human nervous system, and it leaves fingerprints.

Distilled from arXiv cs.AI