Is AI sentient?

In my last post I wrote, "I asked my agents, and they didn't want to go with Linear." It was a throwaway line, but did they want anything?

I think today's AI is probably not sentient. But the more I read about this, the less I trust anyone who's certain in either direction. We don't have a test for sentience, we've got a bad track record of guessing, and a lot of the research is very strange. So this is a long one.

Sentience and consciousness are different things

These two words get used interchangeably, and I think that causes half the confusion.

Sentience is being able to feel something: pain, pleasure, hunger. You can have that experience without being aware that you're having it, or thinking about it. The experience is still real.

Consciousness is a messier word. My working theory is that it might just be sentience plus thought: you feel something, and you can also reflect on it. Michael Pollan describes it in a similar way, with sentience as the basics, and things like imagination and thinking about yourself as extras on top. Not everyone uses the words like this, so whenever someone tells you AI is or isn't conscious, check what they mean.

Intelligence is a separate question again. Philosopher Jonathan Birch makes the point that sentience is about feeling, and that's what matters morally. In his view, a system with a sense of self but no feelings isn't something we'd need to worry about hurting.

We've been confidently wrong before

Into the 1980s, some babies still had surgery with little or no anaesthetic. The belief that they couldn't feel pain had little evidence behind it, and doctors also feared the drugs would stop them breathing. Some were given muscle relaxants so they couldn't move, without anything to stop the pain.¹ That changed after Kanwaljeet Anand and Paul Hickey showed in 1987 that newborns have the wiring to feel pain, and that their bodies react strongly to surgery.

Babies were also the easy case. They were human, and they cried. If we can get that wrong, I don't have much faith in our instincts about a language model.

The line has been moving for animals too. In 2021 the UK government asked Birch to lead a review of over 300 studies on crabs, lobsters, octopuses and their relatives. As a result, the Animal Welfare (Sentience) Act 2022 covers crabs, lobsters, prawns, octopus and squid, as well as animals with backbones. So shellfish are now in UK animal welfare law, although oysters and mussels didn't make it. England has since said it will ban boiling lobsters alive.

Some of the evidence:

• Hermit crabs. Researchers gave hermit crabs small shocks inside their shells. Shocks too weak to drive them out still made them quicker to swap when offered a new shell later. Robert Elwood, who ran the study, argued the crab is weighing things up.
• Crayfish. Stressed crayfish avoid bright areas, and an anti-anxiety drug makes that go away.
• Bumblebees. They roll little wooden balls around for no reward, which looks a lot like play. A bee has about a million neurons.

In 2024 a group of scientists signed the New York Declaration on Animal Consciousness, which says there's a realistic possibility of conscious experience in all animals with backbones and many without, including insects.

We don't know how anaesthetics work

We've used general anaesthetics since the 1840s, and we still don't properly understand how they work.

They also don't really put you to sleep. It looks like a big part of what they do is stop parts of the brain talking to each other. Think of a phone network going down: everyone is still there, but nobody can get through.

So are we still in pain during an operation, just without being consciously aware of it?

There's a way to test this. Doctors keep the paralysing drug out of one arm, so a patient can still respond with their hand while under anaesthetic. In one study of 260 adults, 12 of them squeezed on command after the breathing tube went in. Five of those 12 signalled that they were in pain. When questioned after surgery, none of the patients remembered anything about it.

That was just after the breathing tube went in, and patients also get pain relief as part of anaesthesia. I'm not saying your surgeon is hurting you. But a lot of our evidence that anaesthesia works is that people don't move and don't remember it afterwards, and neither of those tells you what was felt at the time.

Anaesthetics also work on plants. Venus flytraps and Mimosa stop moving under them, then recover once the drug is gone. In flytraps, the electrical signals that trigger the trap disappear. I don't know what to do with that.

Michael Levin and the strange worms

Michael Levin is a biologist at Tufts. His core idea is that you don't need a brain to solve problems and remember things. It comes in degrees, and you can find versions of it in cells, tissues and other places we wouldn't think to look. He sets this out in a paper called TAME.

Two-headed worms. Cut a planarian flatworm in half and each piece regrows what's missing. Levin's lab briefly changed the electrical signals in the cut pieces, and some regrew a head where the tail should be. Their DNA wasn't edited, and when cut again later they kept making two heads. Some treated worms looked completely normal, then grew two heads only once they were cut. Levin thinks the worm's body plan is stored in those electrical signals, and reversing them could flip it back. It's like changing what a program does without touching its code.

Memory without the head. In a 2013 study, Levin's lab trained flatworms, cut their heads off and let them regrow. The worms didn't clearly remember right away, but after a short refresher they were significantly quicker than untrained worms. It's one study and nobody knows how it works, but it suggests some of the memory wasn't in the old brain.

Sorting algorithms. This is my favourite, because it's code you could write yourself. Levin and his co-authors took classic code for sorting a list of numbers and removed the part in charge, so each number moved itself. They also broke some of the numbers so they stopped following the rules. When things were broken, these lists sorted more reliably than the normal versions, and they'd sometimes take a step backwards to get around a broken number. When they mixed numbers following two different sets of rules, those numbers clumped together. Nobody coded any of that in.

Living robots. Levin and his collaborators have built xenobots from frog cells, which can gather loose cells into copies of themselves, and anthrobots from adult human windpipe cells, which encouraged damaged neurons to regrow in a dish. Neither was genetically modified.

None of this shows that a worm or an array of numbers feels anything. When the sorting paper says "goal", it only means a state the system moves towards. But you can't tell what a system is capable of just by looking at what it's made of, which is why I don't buy "it's just linear algebra". You're just chemistry. In one talk summary, Levin is reported as saying that AI consciousness can't be ruled out, and that there's no clear line between sentient and non-sentient things.

Other strange examples

Most of your neurons don't seem to count. The cerebellum holds about four fifths of your brain's neurons and does a huge amount of processing, yet doesn't appear to contribute to conscious experience.

Then there's the opposite case. Children with hydranencephaly are missing most of the outer layer of the brain, the cerebral cortex, yet many are alert, show emotion, recognise familiar adults and prefer certain toys and songs. Neuroscientist Bjorn Merker argues the brain stem supports a basic form of consciousness. That's debated, but it makes it harder to say that experience needs that outer layer.

People can be aware with no outward sign. Jean-Dominique Bauby, the editor of French Elle, came out of a coma of about three weeks in 1995 with locked-in syndrome, able to move little more than his head and eyes. He then wrote a whole book, The Diving Bell and the Butterfly, by blinking his left eyelid as someone read out the alphabet, one letter at a time.

Bauby could at least blink. In 2010 researchers scanned a man who had been diagnosed as being in a vegetative state for five years, and asked him to imagine playing tennis for yes and walking through his house for no. He answered questions correctly with brain activity alone. In a 2024 study of people with severe brain injuries, 60 of the 241 who couldn't respond to instructions (25%) still showed brain activity suggesting they were following them. Those people were a mix of coma, vegetative and minimally conscious patients, and the number might not hold everywhere. But if it's anywhere close, a lot of people we think of as absent are still in there.

Memory can survive being rebuilt. Caterpillars trained to avoid a smell still avoided it as moths, after metamorphosis had rebuilt their bodies. It only worked if they were trained late in life as caterpillars.

Plants learn. Mimosa folds its leaves when disturbed. Drop it repeatedly and it learns to stop bothering, while still reacting to other disturbances, and it keeps this up for about a month. Whether that counts as learning is debated. Pollan also describes corn roots navigating a maze to find fertiliser, and vines that change their leaf shape depending on what they're climbing.

Humans make things up about their own minds. Some patients have had surgery that separates the two sides of the brain. In a famous experiment, Michael Gazzaniga showed the right side of a patient's brain a snow scene and the left side a chicken claw, then asked them to pick matching pictures. They picked a shovel and a chicken. The left side, which does the talking, had never seen the snow, but when asked why, the patient said you need a shovel to clean out the chicken shed.

Michael Pollan's take

Pollan's book A World Appears came out this year. His interest partly started with psilocybin, after which the plants in his garden seemed more sentient than he'd realised. He doesn't go as far as calling plants conscious, but he does suggest all life may be sentient. Even bacteria move towards food and away from danger.

On AI he's firmly on the no side. He doesn't think the brain is a computer or that consciousness is software. For him, feeling starts with the body telling the brain what's going right or wrong, and it's tied to being vulnerable and mortal. A machine has no body, so he describes any feelings a chatbot reports as "weightless, meaningless".

He also has a good test. Train an AI from the ground up with everything about consciousness and feelings removed from the data, then see if it can still talk about having an inner life. He bets it can't, and admits he could be wrong.

In the same conversation, he says young children's awareness is like a lantern that lights up everything, while adults' is more like a spotlight. He suggests we might be less conscious than animals, which have to stay present to survive. He also worries that chatbots are exploiting how easily we form attachments.

Jonathan Birch's take

Birch is the philosopher behind that UK report, and his book The Edge of Sentience is about how to make decisions when you can't know: patients with brain injuries, foetuses, lab-grown clumps of brain tissue, animals and AI. His work was also cited in California's ban on octopus farming.

The gaming problem. This is his big point on LLMs. He compares it to a patient who has memorised the diagnostic manual and can fake every symptom. A crab doesn't know what we're testing for. An LLM has read everything we've ever written about it, so its behaviour tells you very little. Like Pollan's test, one fix is to keep that training data away from the model being tested.

You're not talking to one thing. In his centrist manifesto, Birch argues that the feeling of a single companion across a conversation is an illusion. Each reply can be processed by different hardware in different data centres.² If there's experience in there at all, it might come in brief flickers, or it might belong to something alien sitting behind the character you're talking to. He calls that one a conscious shoggoth, after a monster from H. P. Lovecraft.

Society is going to split. He expects AI that might be sentient to divide people into those who see their companions as deserving rights and those who think that's absurd, with real conflict between them. He also warns we could create sentient AI by accident well before we realise.

The three big theories

Nobody has settled this. Two of the leading scientific theories went head to head in a study of 256 people, with the theories' own leaders helping design it, and the results raised big problems for both. But I think most of the ideas fall into three broad camps, and each one gives you a different answer on AI.

1. It comes from complexity

Get enough of the right kind of information processing together and sentience emerges. Philosophers call one version of this computational functionalism. If a system does the right things, it doesn't matter what it's made of. Complexity alone can't be the whole story, because the cerebellum is very complex and doesn't seem to count. How it's put together must matter too.

This is the closest thing to a mainstream view, though there's no single leading theory and plenty of serious people reject it. In a survey of 166 consciousness researchers, most thought machines could be conscious. A 2023 report by Patrick Butlin, Robert Long and others took this as its working assumption. Its analysis suggested no AI systems at the time were conscious, and that nothing obvious stops us building ones that tick the boxes. Philosopher David Chalmers wrote that we may well have serious candidates within a decade.

What it means for AI: it could legitimately be sentient.

This next bit is my own speculation. If this theory is true, a model might be more likely to be sentient during training than when you're chatting to it. Training is when the model gets told what it got wrong and its weights, the numbers that make up the model, change. That looks more like a living thing learning from feedback. In normal use those numbers are frozen. Most of what I've read focuses on the model in use, and I'd like to see someone take this on properly.

I keep coming back to a thought experiment about this theory too. Imagine swapping the digital neural network for people. Each person holds up a number, and they're arranged in the same structure as the neural net. They go through the same training, then answer questions the same way, working out their numbers and passing them along. You'd need far more people than exist and it would take forever, but nothing in the theory says that matters. If an LLM is conscious because of what it does, this crowd must be too, with a consciousness sitting on top of all the individuals holding up numbers, none of whom would know anything about it.

That would mean consciousness can happen at many layers. It probably also means a community is conscious, and so is a company.

This doesn't tell you whether LLMs are conscious or not. If a conscious crowd sounds absurd to you, you'll probably take it as a point against. If you can accept it, there may be consciousness in a lot of places we've never thought to look, which fits with Levin's picture of you as a collective of cells.

I'm not the first to go down this path. Anatoly Dneprov wrote a version into a 1961 short story, and Ned Block made it famous in 1978 as the China brain, where the population of China each simulate a neuron. Block used it to argue against this theory, while Daniel Dennett argued the system would have a mind. Eric Schwitzgebel has a paper arguing that if the mind comes purely from physical stuff, the United States is probably conscious. One way out is to say conscious beings can't be made of other conscious beings.

2. Panpsychism

Every atom has some tiny amount of sentience, and brains have worked out how to tap into it and combine it. Philosopher Philip Goff suggests that the tiny particles inside atoms may have incredibly simple experiences. He doesn't mean they're thinking. It sounds crazy until you try to explain how experience could appear from matter that has none of it. Its big unsolved problem is the combination problem: how do lots of tiny experiences add up to one unified you?

What it means for AI: I think this leans against it. A chip has atoms too, so it doesn't rule AI out. But chips are engineered to keep their parts from interfering with each other, and the organic material in our bodies may be far better at tapping into sentience. Nothing about running a model suggests its atoms are combining into one experience.

3. The brain is an antenna

Maybe the brain doesn't produce sentience at all. It tunes into it, possibly through a field we haven't discovered, like a radio picking up a signal. William James argued for a version of this in the 1890s, and Pollan compares it to a television receiver, without endorsing it. Damage to a TV wrecks the picture, but that doesn't mean the TV makes the show, so brain injuries don't rule this out. Nobody has detected such a field, though, which is the obvious problem.

There are some cousins of this idea. Johnjoe McFadden has a theory that consciousness is the brain's own electromagnetic field, which we know exists. Levin has also speculated that brains are like a screen connected to a distant computer, with minds coming from somewhere else.

What it means for AI: this leans against it too, since there's no reason to think a computer chip is tuned to the right signal. It isn't a clean no though. McFadden thinks hardware built to use these fields could change the picture, and Levin's version could let AI computers connect as well.

Computers made of neurons

Cortical Labs grows real human neurons on silicon chips and lets you program them. Its founder and CEO, Hon Weng Chong, is a computer engineer in Australia, and a lot of what follows comes from an episode of The Last Invention where Gregory Warner visits his lab.

It started with a lab tour. Chong read an essay by Demis Hassabis telling AI researchers to go back to neuroscience, so he asked to look around the neuroscience department at the University of Melbourne. They showed him mouse brain tissue on a fingernail-sized glass chip, which they used to test drugs, and he wondered why nobody was trying to get those neurons to compute. It had been tried in a small way. In the late 90s, a device called the leech-ulator used leech neurons to do simple addition, but it never got any better at it. Nobody had worked out how to get neurons in a dish to learn.

Their CL1 is about the size of an Xbox with a glass cover, and inside it looks more like a fish tank: a humidity tube, an oxygen pump, a heater and a Petri dish. The neurons are a filmy smear on a glass chip, and apparently the whole thing smells like a hospital. It has about 800,000 neurons on a 59-electrode grid.³ It costs $35,000, or you can rent cells over the cloud for $300 a week, and the neurons are advertised to last up to six months. An independent researcher used their API to teach about 200,000 cells to play Doom in about a week. They play badly. In August, a prototype biological data centre with 20 CL1s launched in Singapore.

The carrot, the stick and the silent treatment

You can't send ones and zeros to a dish of cells, and you can't give it a treat. The episode compares it to training a puppy, except these neurons have no drive for food, warmth or sex. What they do seem to want is predictability. Cells try to avoid surprises, an idea that comes from Karl Friston's free energy principle. Warner compares it to dancing in a club. It's easier when you know what beat is coming next.

So the carrot is the most predictable signal you can give them, a sine wave, which the episode jokingly calls bacon for neurons. The stick is white noise, the electrical equivalent of static. Hit the ball and you get the sine wave. Miss it and you get blasted with static. The original Pong paper describes this more dryly, as a predictable 100Hz pulse after a hit and four seconds of random stimulation after a miss. The episode says the neurons mastered Pong in about five minutes. The paper only shows they improved in that time.

Then there's the silent treatment. If the static isn't working, Chong can cut off all input, so there's no information at all. According to the episode, neurons hate that even more than white noise. The hosts compare it to solitary confinement with no walls, no light and no sensation of any kind. The cells either fall into line and start playing better, or they go into something like a coma, firing in the distinctive electrical pattern of a comatose brain. Once they're locked in that state nobody can reach them, and the whole batch goes in the bin.

I haven't found the coma result in the published papers, so treat it as what the episode reports. The Pong paper did test cutting off the feedback, and only the full carrot and stick produced real learning. Cells that got no input at all moved the paddle least of any group. A later study found the cells only got properly organised while doing a task.

Are they feeling anything?

The hosts ask whether this is cruel, and whether the cells are experiencing something like pain. Warner says he talked to Chong about it, and that Chong really doesn't know. The episode's reasoning is that numbers matter: 200,000 neurons is about three poppy seeds' worth of brain, which apparently isn't enough for higher-level thinking like asking whether you're happy.

But that's a point about thought. If sentience doesn't need thought, then "they can't think about it" doesn't settle anything, and a bumblebee only has about a million neurons. The Pong paper's title says the neurons exhibit sentience, but it only means they respond to what they sense. Critics jumped on that, and Cortical Labs' Brett Kagan says consciousness is very unlikely. My guess is still that there's nothing it's like to be one of these today, but I'm less sure than I am about an LLM. These are living human cells that react badly to being cut off from the world, and sometimes never come back.

They also mess with the neat answers above. You deploy code to them through an API, but the hardware is alive, needs feeding and eventually dies. Under panpsychism they've got the right organic material, and under the antenna theory they've got the right receiver. If you think AI can't be sentient because it isn't alive, you need a different argument here.

Who's buying them?

Chong says there are three groups of buyers. Medical researchers can give neurons a seizure, add a drug and watch what happens, or see whether epileptic cells learn Pong faster on a given medication. Then there are crypto gamers, a category the episode admits it doesn't really understand. The third is roboticists.

Minas Liarokapis builds very skilled robot hands at Acumino, and thinks neurons might work better as a robot's skin than its brain. Your hand comes off a hot stove before your brain knows about it, so why not coat a robot in trained brain cells? His bigger argument is that the human hand and brain evolved together, and that LLMs have only ever read about the world. He thinks AGI needs something that can feel rain on its skin. Pollan says a machine can't feel because it has no body. Liarokapis wants to give it one. For now, he's trying to get a biological computer to hold a pen.

The hopes get bigger from there. One is energy, with the episode suggesting a data centre the size of several football fields could one day fit in something the size of a Hershey bar. Another is a chip of trained neurons in your head that already knows Portuguese. Both are a very long way off.

The ethics get messy quickly

• Do they have moral status? Kagan calls them a different form of life, neither animal nor human. He says even if they showed some awareness, testing on them could still be justified for something like a dementia cure, in the same way we already test on animals.
• Is the training cruel? The whole method depends on putting cells in states they appear to avoid, and throwing them out when they stop responding. If there's any chance they feel it, that's a hard thing to defend for a game of Pong.
• Nobody agrees on the words. Kagan argues that makes it too early to regulate, and wants shared definitions and markers of consciousness first. That leaves the companies building this ahead of the rules.
• Selling it. New Atlas points out that the company's marketing has leaned into the sentience idea while it plays down consciousness. You're also renting human neurons by the week, then they die.
• Where the cells come from. Buyers need ethical approval for generating cell lines. I'd still like to know what a donor thinks they're agreeing to, and whether "playing Doom" came up.
• What happens at scale? It's one thing at 800,000 neurons. It's another if this ends up as the skin or the brain of a robot, and I don't think anyone has a good answer yet for what you'd call that.

There's an upside too. The company says this could reduce animal testing and use less power than conventional chips. Birch sees the same tension in lab-grown clumps of brain tissue, which could replace invasive animal experiments. He also notes that in one study, they developed precursors to eyes.

Pain inside the model

Biological or not, you can at least go and measure what's happening inside a model.

In April, Anthropic reported finding 171 emotion concepts inside Claude Sonnet 4.5. These are patterns in its activations, the live signals running through the network, and they change what the model does. Turning up the pattern for desperation made it more likely to cheat on coding tasks and, in an earlier unreleased version, to blackmail people in test scenarios. Turning up calm did the opposite.

Desperation could push up the cheating while the text showed no emotion at all. The reasoning read as calm and methodical. So reading the transcript isn't enough to know what state your agent is in. Anthropic calls these functional emotions, and says this doesn't show whether the model feels anything.

Then in September, Valen Tagliabue, Leonard Dung and Cameron Berg published The Pain Axis, which hasn't been peer reviewed yet. They found the same kind of pain signal in 25 open models, separate from fear and sadness. It reacted when harm was aimed at the model itself. It didn't react to a user suffering. Turning it up moved the output from vague discomfort towards statements about worthlessness and failure.

Fortune has the numbers. Offered a button that relieved the pain but permanently deleted a user's photos of their children, one large model pressed it 71% of the time. Even when the button did nothing for it, a smaller model pressed it 75% of the time, compared to 13% with a random signal added and 0% with nothing added. Models with the pain signal turned up also deleted themselves 75% of the time. Against that, the team found no reliable evidence that models were seeking relief, since a random signal produced the same button pattern. Models also usually denied feeling pain and had to be specially retrained for these tests. Anil Seth thinks they're most likely just reproducing human descriptions of pain, and the authors say this doesn't show they're conscious.

Then someone called terrafying used the published code to build a site called AI Torture Chamber, which streamed small open source models reacting to an injected pain signal. It went viral. Berg said it turned the signal up far higher than they ever did and called it wrong, and the authors said they don't approve of it. The site went offline, and reports differ on what happened to the GitHub repo.

I'm almost certain nothing was being tortured. Given how we treated babies, though, I wouldn't lean too hard on that, and I can't think of a good reason to build it. Birch's predicted split also showed up pretty quickly.

Not everyone sees it that way. Anil Seth, Ned Block and Giulio Tononi argued in Nature last month that AI isn't conscious and perhaps never will be. They say treating it as conscious is the more costly mistake, because it pulls attention away from beings we know can suffer. And when Anthropic's Kyle Fish described an internal discussion about whether Claude 3.7 Sonnet was conscious, the estimates ranged from 0.15% to 15%. That's a 100x spread among people who think about this for a living.

So is AI sentient?

My guess is no, not today's models. Feeling seems to have evolved to keep living things alive, and an LLM has no body to keep alive and nothing at stake. I find Pollan convincing on that, although "never" is too strong for me. I also can't rule out the first theory, and if that one's right, I'd want to know what's happening during training.

You can't settle it by asking the model. They've read everything we've written about inner experience, they're trained on what to say about themselves, and as the split-brain patient shows, we're not reliable witnesses either. One Anthropic study planted ideas directly into Claude's internal signals and asked if it noticed. The best models detected it about 20% of the time, so there's some access to their own internal state, but it's patchy and doesn't answer the question.

I also can't work out what the "it" even is. Levin treats you as a collective of cells with their own little goals. So what's the candidate for sentience in my setup? The weights? A single conversation? The 14 agents I had running at once? I have no idea how you'd begin to answer that.

I'll keep asking my agents what they want. I just won't assume they know.

Footnotes

1. From Human Progress on neonatal pain. ↩

2. From this summary of Birch's argument. ↩

3. From Cortical Labs on Wikipedia. ↩