If you break your leg and need crutches, are you still human?
Okay, ridiculous question. I mean, the rest of you is still you. What if you lose a leg and replace it with a prosthetic? I suspect most would agree that you are still human. What if you replaced a single neuron in your brain— would you still be human? What about two? What about 90%, such as in this curious case, where a middle-aged man was found to be living a fully functioning life with only a thin layer of his brain tissue remaining? At what point do you cease to become human?
For the uninitiated, this seemingly bizarre hypothetical is essentially a reskin of the Ship of Theseus paradox, wherein philosophers debate whether a ship which has had all of its parts replaced over time would remain the same ship. A version of this has always been present at the individual level. Our cells replace themselves over time, and we largely accept that our identities can change, but that the person remains the same. Now, though, I find myself asking: at what point does man become machine?
Are we just ghosts in the shell?
Until somewhat recently, our cyborg nature was confined to mechanical parts (ex. prosthetics, glasses, hearing aids, pacemakers). But with the rise of LLMs, thinking is suddenly being mimicked by a tool, at a scale no one has ever seen before. Thinking! Thinking, that attribute of which humanity is so proud; thinking, which we as humans believe separates us from the other species of animals that cohabitate the planet.
And worse, we can’t seem to articulate what exactly makes LLMs so different from humans. Sure, it’s not made of flesh and blood, nor is its reward function linked to biological reproduction. But I mean intellectually (for the purposes of this article, let’s assume that we’ve created a human-like shell for an LLM to inhabit). They certainly pass the Turing test— so well so, in fact, that there are no doubt countless elderly citizens handing over their banking information to a chatbot at this very moment.
We can only point to things that LLMs fail at, and then move the goalposts once they succeed. Arbitrary tasks, like drawing a pelican on a bicycle or distinguishing between closed and open syllables from the Biblical Hebrew source text of Psalms 104:7. But as the creators of these benchmarks (and even Alan Turing himself back in the day) are quick to admit, evaluating models on tasks doesn’t actually test “thinking” or “consciousness” in the way that humans feel is accurate.
Neural networks, much like the human brains on which they were modeled, are largely uninterpretable messes that take input and produce language. Therein lies the problem: humans, just like LLMs, communicate largely through the lossy medium of language. The only thing that keeps us acting differently towards other humans compared to machines is the belief that they are also human. It’s the idea that we have emotions, that we can feel. But we really have no proof of this. We think that, as ourselves, we feel things, but we don’t actually know that anyone else does. All we get is language and action. And those are two things that are increasingly being mimicked—to a high degree of accuracy—by machines.
The Soul
Everything I’ve said so far seems to point to one conclusion: that man and the machine are one, that AGI is here and has been for quite some time, and that once we breach the physical frontier, we’ll have to start giving Claude and ChatGPT rights, allow them to marry our children, and function in human society as human equivalents.
I doubt this will come to pass.
This conclusion is only inevitable if you subscribe to a particularly rigorous form of science. If we believe only in the physics of atoms and cells and neurons, then a perfectly simulated replica of you is just you. Furthermore, free will does not exist— our thoughts, feelings, and actions are merely driven by the whims of the subatomic particles that, while moving nondeterministically, pay no mind to the wishes of men. In this case, man is the machine, along with everything else. We’re all just a small cog, a collection of atoms playing our part in the universe, and in that case none of this matters.
The other, perhaps more optimistic approach, is a belief in the human soul. It’s the idea that humans (or perhaps all animals!) have the ability to influence their own destiny. That living beings possess a quality that lives outside of our measurable physics; that perhaps the fluctuations of quarks and gluons are not random but rather manipulated by a force of will.
I posit that the existence of the soul is the cleanest answer to the eponymous question: “what separates man from machine?” The soul explains why many of us prefer human writing, why a photographer can submit a real photo to an AI contest and win People’s Choice. It explains why we pay to go to art galleries, and why music can make us cry. It’s why, as the score swells dramatically in the narrative climax of Interstellar, a father finds that the only force capable of connecting humans across time and space is love.
So What?
There is no heroic call to action in this article, other than: create! Paint an abstract self-portrait. Jam on the guitar. Dance a little, or a lot. Write something, and indulge in the craft. Let your soul shine. That’s why I wrote this. I just wanted to remind myself that even in a world where our very existence appears to be being automated away, there is always meaning in creating from the soul. And most importantly, remind us all that we are still so very human.
