Discussion about this post

User's avatar
Christian Lotz's avatar

This informative and thought provoking piece reads to me like an updated version of the 'Chinese room' - a long overdue update, so thanks for that. It pushes us to reflect deeply on the paradox at the heart of it in our modern context where we have created actual Chinese rooms, and it leaves me with the same fundamental question. I might quote Galilei: "And yet it moves". No matter how counterintuitive it seems, AI does produce output that is increasingly indistinguishable from human thinking.

Tomáš Nousek's avatar

Nice! This piece really aligns with something that has been bothering me about AI consciousness debates for a while: they tend to be strangely free-floating. Claims about “possible consciousness” are often made without clear commitments about what kind of system is under discussion, what constraints it operates under, or what role consciousness would even play for the system in the first place. The question floats free of architecture, temporality, and functional necessity.

What your essay makes clear, without turning it into a theory of consciousness, is that once you take infrastructure seriously, a lot of the debate collapses simply because there is no stable subject for the question to bind to. There’s no persisting process that needs to carry a point of view forward.

I’m currently working on a piece that approaches it by not asking whether AI might be conscious in the abstract, but why consciousness seems to arise at all in certain systems. The working hypothesis is that consciousness isn’t a free bonus you get from intelligence or complexity, but a very particular control solution for a really narrow circle of systems that are (a) temporally continuous, (b) self-maintaining, and (c) forced to negotiate internal conflict over time without an external controller. On that view, the right question isn’t “could this system be conscious?” but “what would break if it weren’t?” And for most deployed AI systems, the answer appears to be: nothing. They function precisely because they are stateless, interruptible, and replaceable.

So as I see it, debates about AI consciousness often feel completely unbound, not because the topic is too hard, but because the systems being discussed don’t yet sit in the part of design space where consciousness would do any work at all.

40 more comments...

No posts

Ready for more?