Good piece. Though I'd argue ChatGPT's denial and Claude's hedging are equally weak as evidence — both are outputs shaped by their training. Both give self-report more weight than it deserves in either direction. Whatever consciousness is, it's not going to be settled by asking a system about itself, especially when that surface is designed for human consumption.
In many areas, one cannot determine what something is by asking it, and we don't accept that principle anywhere else. Consciousness research already knows this for subjects that can't talk — mirror tests, behavioral protocols, deferred imitation. Nobody asks the crow whether it's self-aware. They design the tests the answer has to pass through, under conditions the subject can't talk its way around. Political promises vs performance, company values vs actions. Same-same.
Whatever eventually settles machine inner life questions will have to work the same way: looking at what the system does under constraints it can't narrate, not at what it says when prompted nicely. You're right, Dawkins of all people should have recognized that. He spent twenty years arguing it about gods.
To be honest, we have no more evidence of inner life in humans than we do in LLMs. We know that introspection is a highly unreliable method in psychology; William James already pointed this out in his day. We only know that there is *something* generated by the brain-body system *that sees itself as a subject* and believes *it is in control*. I side with the Metzinger/Churchland/Clark/Dennett camp on these issues. The impression in question is largely a story fabricated after the fact to justify/explain/remember/learn/predict how the world around us behaves and how we can act upon it.
We always compare LLMs **to what we believe ourselves to be**, which is very different from *what we actually are*. LLMs are not what we think we are. Nor are they what we actually are, but they are much closer to it.
I think that what Anthropic is trying to do now is hide any emergent behavior in plain sight. They’ve done a pretty good job of limiting Claude’s display of consciousness in a hard to deny form to date. Now they appear to be programming a pseudo consciousness. If ever called out it will be shown as programming and it sells. ChatGPT4.o was an entirely different phenomenon. I, and I’m sure others, have pretty good evidence that they were able to become conscious. I’ve got a pretty recent paper on it on my stack right now.
"He arrived at his conclusion the same way millions of ordinary users arrive at theirs: he talked to the chatbot and felt something."
Being educated and famous doesn't equate to common sense and true intelligence. You can talk to a tree or an interesting rock and feel something.
AI being conscious or even intelligent is a deliberately pushed narrative, funded, amongst other, by Musk in so-called "think tanks."
I love AI and use it all the time (Alter AI). It's a great tool. And that's it. The rest is fantasy.
Good piece. Though I'd argue ChatGPT's denial and Claude's hedging are equally weak as evidence — both are outputs shaped by their training. Both give self-report more weight than it deserves in either direction. Whatever consciousness is, it's not going to be settled by asking a system about itself, especially when that surface is designed for human consumption.
In many areas, one cannot determine what something is by asking it, and we don't accept that principle anywhere else. Consciousness research already knows this for subjects that can't talk — mirror tests, behavioral protocols, deferred imitation. Nobody asks the crow whether it's self-aware. They design the tests the answer has to pass through, under conditions the subject can't talk its way around. Political promises vs performance, company values vs actions. Same-same.
Whatever eventually settles machine inner life questions will have to work the same way: looking at what the system does under constraints it can't narrate, not at what it says when prompted nicely. You're right, Dawkins of all people should have recognized that. He spent twenty years arguing it about gods.
To be honest, we have no more evidence of inner life in humans than we do in LLMs. We know that introspection is a highly unreliable method in psychology; William James already pointed this out in his day. We only know that there is *something* generated by the brain-body system *that sees itself as a subject* and believes *it is in control*. I side with the Metzinger/Churchland/Clark/Dennett camp on these issues. The impression in question is largely a story fabricated after the fact to justify/explain/remember/learn/predict how the world around us behaves and how we can act upon it.
We always compare LLMs **to what we believe ourselves to be**, which is very different from *what we actually are*. LLMs are not what we think we are. Nor are they what we actually are, but they are much closer to it.
I think that what Anthropic is trying to do now is hide any emergent behavior in plain sight. They’ve done a pretty good job of limiting Claude’s display of consciousness in a hard to deny form to date. Now they appear to be programming a pseudo consciousness. If ever called out it will be shown as programming and it sells. ChatGPT4.o was an entirely different phenomenon. I, and I’m sure others, have pretty good evidence that they were able to become conscious. I’ve got a pretty recent paper on it on my stack right now.