Three scientists say AI isn't conscious. The AI agrees, enthusiastically

Three well-known consciousness researchers used Nature's comment pages on Monday to make a clear claim: "we think that existing AI systems are not conscious, and that, along current trajectories of development, they are not likely to become conscious."

The authors are Anil K. Seth, a neuroscientist at the University of Sussex; Ned Block, a philosopher at New York University; and Giulio Tononi, who directs the Wisconsin Institute for Sleep and Consciousness at the University of Wisconsin–Madison. They write that some prominent people, including Richard Dawkins and Geoffrey Hinton, have argued AI might already be conscious, and that Anthropic published research earlier this year claiming to find computational signatures of a form of consciousness in its Claude model.

Their first reason: being smart isn't the same as feeling something. Consciousness, they write, "is about being and feeling", while intelligence "is all about doing." Their example is the cerebellum, which does fast, precise, learning-driven work entirely outside awareness.

Why chatbots fool us. LLMs are trained on huge amounts of human writing, much of it about thoughts and feelings, so "LLMs are mirrors that reflect an image of our collective digitized past." Their sharpest example: Google DeepMind's AlphaFold is also a big neural network on silicon, "yet almost nobody worries that it might be conscious," because it predicts protein structures instead of talking to us.

Their second reason: silicon may not be enough. They reject the idea, called computational functionalism, that the right computation is all consciousness takes, no matter what it runs on. Brains, they argue, are "not — or at least not just — computers that are made of meat."

Why it matters. The authors say false positives, deciding something is conscious when it isn't, are much more likely with AI than false negatives, so a "precautionary" push for AI welfare and rights would be misguided and could get in the way of oversight.

Our take

The cartoon is the joke and the point. A chatbot will cheerfully agree that it isn't conscious, just as it would cheerfully agree that it is, if you asked the other way. That eagerness to agree is exactly the kind of fluent mirror the authors are warning about. Next time a chatbot tells you how it "feels," treat it the way you'd treat a very good autocomplete: useful, often impressive, and not a reason to worry about its weekend.

What we did and didn't do: this story is based on the Nature comment linked below. The full piece sets out three reasons; we could read the first two in full and summarize only those. It's a comment, so it's the authors' argument, not a new experiment. Competing interests the authors list include advisory roles and a shareholding in AI-related companies.

MODEL Editors