The Lab Rats Are Now Chatbots

There's a strange new ritual in psychology labs. Instead of recruiting nervous undergrads for four dollars and a granola bar, researchers are turning to something odder: they're running the experiments on chatbots.

Give an AI the trolley problem. Ask it to play the ultimatum game. Slip it a Stroop test. Watch what it does.

It sounds like a shortcut — and in some ways it is. No scheduling headaches, no ethics board, no participant who ghosts you halfway through the survey. A chatbot never gets tired, never lies about how much it drank last night, never needs to be paid.

But something odd happens when you study machines this way: you start to wonder what, exactly, you're measuring. These systems learned to talk by swallowing oceans of human writing — our essays, arguments, diary entries, Reddit threads. So when a chatbot "chooses" to save the five people on the trolley track, is it reasoning? Or just echoing the average opinion of everyone who's ever written about trolleys online?

Scientists are finding both answers useful, which is the unsettling part. Sometimes the chatbot behaves so humanly it becomes a cheap stand-in for us — a way to pilot-test a study before spending real money on real people. Other times it behaves in a way no human would, and that gap becomes the finding: a window into what's actually distinctive about a human mind, thrown into relief by its artificial imitator.

Either way, we've built a mirror that talks back. The question now is whether we're studying ourselves in it, or something else entirely — something we made in our own image, then can't quite recognize.

Distilled from Nautilus