AI Is Not Conscious, But It Is Becoming Our Unconscious

The Convivial Society: Vol. 7, No. 4

Welcome to the Convivial Society, a newsletter about technology, culture, and the moral life. I’ll tell you upfront that this is an odd piece in which I try to write my way through an intuition that the role of AI in society can be helpfully framed, at least in part, by analogy to the unconscious. Whether or not you buy the whole thing, I trust you’ll find at least a few helpfully provocative considerations along the way or at least some helpful questions to think with. However useful or not these reflections might be, I offer them out of the conviction that we need better, more fruitful ways of grasping these strange new technologies in our midst and the nature of our relation to them.

Cheers,
Michael

---

1. According to the early 20th-century English mathematician and philosopher Alfred North Whitehead, “It is a profoundly erroneous truism, repeated by all copy-books and by eminent people when they are making speeches that we should cultivate the habit of thinking of what we are doing.” “The precise opposite is the case,” he argued. In his view, “civilization advances by extending the number of important operations which we can perform without thinking about them.” He concluded with a rhetorical flourish: “Operations of thought are like cavalry charges in a battle — they are strictly limited in number, they require fresh horses, and must only be made at decisive moments.”

There’s a good chance you’ve seen this quoted at some point. It’s fine. It conveys some bit of truth, I’m sure. Nonetheless, it has always struck me as somehow shortsighted or inadequate. Perhaps it’s because I tend to see these lines from Whitehead quoted in defense of indiscriminately outsourcing human cognitive activities to machines without a proper accounting or even awareness of the attendant costs. In defense of Whitehead, who hardly needs my defense, it may also be because those who are quoting him in this manner are almost certainly extending the force of his argument beyond the scope he intended. After all, that paragraph comes from his 1911 An Introduction to Mathematics and what he is actually talking about in that section is the advantage of symbolism and notation in facilitating mathematical calculations, and while he characterizes these as allowing for operations performed without thinking, they are nonetheless learned and deployed by the thinking mind. That said, his talk of “civilization” and that last rhetorical flourish probably invites such a (mis)reading.

The question is not whether such automations of thought, or even externalizations of certain mental processes, can be useful in certain cases. For what it’s worth, I actually think you get a more compelling argument by analogy in the realm of physical rather than mental activities. I’m many years removed from whatever athletic skill I might have once possessed, but the lessons are not lost to me. Much to the dismay of almost every young athlete taking up a sport for the first time, you must spend what always seems like an inordinate and excruciating amount of time executing basic and repetitive drills, over and over again. Wax on, wax off, for those of you of a certain age. But what is, in fact, happening is an ideal example of the dynamic Whitehead is describing. You are in effect automating certain physical movements so that you can perform them without having to think about them. Only then can you play with any kind of creative or exceptional skill. The same holds for learning to dance or to play a musical instrument, etc. Automating basic physical motions is the indispensable foundation of virtuosity.

But does this dynamic apply equally to all realms of human activity? Are there cases in which outsourcing certain forms of activity undermines rather than enables the achievement of the higher goods for the sake of which the activity is pursued? Or might there be goods that attend the “lower order” activities that we would not want to do without? Or is it even always possible, as in mathematics perhaps, to so easily disambiguate distinct sub-routines from given processes or activities? Are there not any irreducibly integral activities that would not survive contact with an attempt to outsource or automate any of their elements? And is there not a difference, as I suggested above, between internally mastered automations of thought and the outright externalization or wholesale outsourcing of cognitive labor? To return to the analogy to physical activity, the would-be athlete that hypothetically employs a machine to do all the “menial” drills for them so that they can get to the really exciting parts of the game will, in fact, never get to them at all. These all strike me as vital and critical questions to explore before we assent to the promises of efficiency and liberation so readily made on behalf of novel technologies.

2. It may be, however, that I’m also rankled by Whitehead’s oft-cited celebration of cognitive automation because it appears to starkly contradict Hannah Arendt’s admonition that we “think what we are doing.” Whitehead was writing nearly half a century before Arendt, and I don’t think Arendt was in any way alluding to Whitehead. It just so happens that these two claims rhyme antagonistically, and I find it useful to explore the tension.

“It is a profoundly erroneous truism … that we should cultivate
the habit of thinking of what we are doing.”
— Alfred North Whitehead

“What I propose, therefore, is very simple: it is nothing more
than to think what we are doing.”
— Hannah Arendt

Interestingly, Arendt’s admonition comes from a context that is much closer in its concerns to our present anxieties about technology and human existence. This line comes from the Prologue to The Human Condition, which was first published in 1958. “What I propose in the following,” she explained,

“is a reconsideration of the human condition from the vantage point of our newest experiences and our most recent fears. This, obviously, is a matter of thought, and thoughtlessness—the heedless recklessness of hopeless confusion or complacent repetition of ‘truths’ which have become trivial and empty—seems to me among the outstanding characteristics of our time. What I propose, therefore, is very simple: it is nothing more than to think what we are doing.”

Among the “newest experiences” eliciting Arendt’s investigations were the launch of Sputnik and the advent of automation. While it may have until recently been judged that Arendt’s fears regarding automation’s impact on labor were misplaced, it might turn out that they are better judged to have simply been premature.

3. As I consider the difference between Whitehead and Arendt on the question of whether we need more thinking or less, I find it useful to distinguish between, on the one hand, building a repertoire of non-conscious routines that allow you to develop higher capacities and, on the other, being in thrall to unconscious forces which erratically drive your behavior in a potentially destructive manner.

Is there a threshold across which Whitehead’s principle flips or reverses? “Civilization advances,” Whitehead claimed, “by extending the number of important operations which we can perform without thinking about them.” But what if there is a tipping point? Past a certain threshold, either of quantity or quality, does civilization degrade by extending the number of important operations humans can perform without thinking about them? Or at the very least, do we get a very different sort of civilization than the one we have known? We may soon learn the answer to these questions because the adoption of AI often amounts to the automation of our thinking and the creation of a vast realm of non-conscious action in the world that arises from the externalized storehouses of our personal and collective memory.

4. Arendt worried that we might lose the ability “to think and speak about the things which nevertheless we are able to do” as our technological capabilities, explicable chiefly in the language of mathematics, outstripped our capacity to comprehend them in ordinary language. (This suggests that a form of the problem of interpretability predates the advent of LLMs and so-called blackbox algorithms.) In a passage that has renewed relevance and urgency, she went on to say that “it would be as though our brain, which constitutes the physical, material condition of our thoughts, were unable to follow what we do, so that from now on we would indeed need artificial machines to do our thinking and speaking.” This was a political problem of the first order. Under such conditions, we would no longer be, in any meaningful sense, governing ourselves. Politics as a realm of human action, and constituted by speech, would cease to exist.

5. On Christmas Day, 1958, the same year that Arendt published The Human Condition, W. H. Auden, published “Friday’s Child,” which included these thoroughly Arendtian lines:

The self-observed observing Mind
We meet when we observe at all
Is not alarming or unkind
But utterly banal.

Though instruments at Its command
Make wish and counterwish come true,
It clearly cannot understand
What It can clearly do.

Auden and Arendt were friends and Auden favorably reviewed The Human Condition, but I am uncertain as to the direction of influence.

6. While Arendt offers a straightforward account of how we arrive at the place where we clearly cannot understand what we can clearly do, that is to say, when our technologically enabled action cannot be fully comprehended by our ordinary language, there is another intriguing way of framing the matter.

Actions that are impenetrable to language and thus to conscious thought can also be explained by reference to the unconscious. I am not one to instinctively appeal to the language of the unconscious, but it may be a useful analogy in the service of understanding our relation, individually and collectively, to the artificially intelligent apparatus that is increasingly mediating our experience of the world.

To put it plainly, I can think of two ways of analogizing certain instances of consumer AI to the unconscious. The first is relatively straightforward: as we outsource more and more tasks at the personal, organizational, and institution levels of society to agentic AI, we generate a layer of activity in the world that is functionally sundered from active human judgment and oversight.¹ And to the degree that this layer of society structures and informs our experience, the ratio of conscious to unconscious human action shrinks.

Obviously, since at least the dawn of the industrial age if not earlier, there have been relatively opaque systems at work in the human lifeworld. The difference, as I see it, is that these systems were relatively sequestered from the course of ordinary human activity. Artificially intelligent systems on the other hand are increasingly woven far more intimately into our experience. In many cases, these are not only processes that are brought to bear on us by external forces, they are also processes that we unleash and initiate and which act for us and back on us. It is this proximate intermingling of human and machine that leads me to reach for the analogy to the unconscious.

Erik Hoel recently made a similar case more eloquently and at greater length. Here is his conclusion:

The traditional danger of AI is usually thought to be superintelligence acting as an existential threat. Yet, this may miss the true and more subtle danger: the AI revolution is a mechanism for transferring the processes of our civilization from under the supervision of consciousness to unconsciousness. But as AI removes consciousness from the workings of the world, it renders the world increasingly uninterpretable, ever more strange and unintelligible. So far, the great ensloppification of the commons has supported this as the major risk of the LLM revolution. And as AI systems become more intelligent, especially if they remain (or are likely to remain) non-conscious, then a further significant risk is consciousness receding in cultural importance.

7. This line of thought runs parallel to an argument I’ve made on and off for about ten years now: digital technology alternatively re-enchants the world. Which is to say that it renders the world mysterious and inscrutable. More:

Our technologically enchanted objects confront us with meaning that imposes itself on us and with which we must reckon. We turn to our technologies for help and invest our hope in their power. We also fear our technologies and see them as the cause of our troubles. The technological forces we encounter are sometimes benevolent but just as often malevolent forces undermining our efforts and derailing our projects.

It is not only that technological objects have the potential to empower us and sometimes even fill us with wonder. It is also that we experience these objects and forces as important determiners of our weal and woe and that they act upon us independently of our control and without our understanding. We are, in other words, vulnerable, and our autonomy is compromised by the lines of technologically distributed agency that intersect our will and desires.

8. The second way of conceiving AI by analogy to the unconscious is perhaps a bit more esoteric, so bear with me. In Understanding Media, Marshall McLuhan claimed that “with the arrival of electric technology, man extended, or set outside himself, a live model of the central nervous system itself.” At several points, he returns to this idea of electric media as an externalization of our nervous system. For example:

“It is a principal aspect of the electric age that it establishes a global network that has much of the character of our central nervous system. Our central nervous system is not merely an electric network, but it constitutes a single unified field of experience.”

McLuhan was not the first to suggest as much.² As early as 1950, the Catholic priest and paleontologist Pierre Teilhard de Chardin claimed that media technology constituted “the creation of a true nervous system for humanity” and “the elaboration of a common consciousness.” Through the emerging network of computational and communication technologies, a new layer of reality was emerging, one that enveloped the biological. It was a layer of technologically mediated, unified human consciousness, which he called the noosphere and described as “a stupendous thinking machine.”

(I confess that this is not exactly my customary idiom, but let’s play with these concepts just a little bit. As it is, I’m already trying to convince you that approaching AI psychoanalytically can be a helpful move.)

Electric media extended our capacity to perceive the world, bringing far-flung events to our eyes and ears and collapsing the time it takes to convey a message to near instantaneity. But something else has emerged since the time of Teilhard de Chardin and McLuhan: the enormous growth of artificial memory or data storage capacity. So while electric media extends our nervous system, it more recently also facilitates the collection and storage of unprecedented amounts of information in the form of texts, images, videos, etc. Our nervous system has been supplemented with a digitized memory of gigantic proportions. And it is this digitized memory which has been instrumental in feeding the Large Language Models that are now nearly synonymous with artificial intelligence.

This enormous repository of human knowledge and culture is basically the internet, or at least the internet is how it becomes accessible to us. It is for this reason that in the early 2000s the theorist of digital media, Gregory Ulmer, referred to the internet as a “prosthesis of our collective unconscious.”

Here’s what I think Ulmer means by this. We have always been social animals, that is to say that we are intersubjective creatures of culture. But most of what has shaped us, perhaps especially in our childhood, has also usually faded from our conscious awareness over time. However, the internet functions as a prosthesis of this unconscious dimension of our formation as social beings by making our collective cultural memory accessible. The internet is vast web of the cultural artifacts, symbols, images, texts, and diverse ephemera that have shaped us throughout our lifetimes, linked not by a linear logic but by an almost dreamlike logic of association, not unlike the human unconscious in psychoanalytic theory.

Ulmer believes that this presents us with a profound opportunity. By making the collective unconscious something that we can reflect upon, something accessible to conscious thought, we thereby have a way of overcoming what would have ordinarily been a profound blindness at the heart of our experience. Needless to say, I’ve always been less than sanguine about this possibility. But regardless of your estimation of Ulmer’s optimistic vision—which I have considerably, although I hope not unfairly abridged here—it does seem to me that the emergence of commercial chat-based AIs takes us in a very different direction.

Interestingly enough, the insertion of an ordinary language interface between ourselves and the digitized collective unconscious makes it more obscure and inscrutable to us. The chatbot interface reconfigures our agency in navigating the collective unconscious by, in a manner of speaking, becoming an anti-therapist leading us away from self-knowledge and insight, however disturbing or startling, toward a comfortable and soothing encounter. It offers a false clarity and lulls us into self-satisfaction, guarding us from self-doubt and from lingering too long in an awareness of our ignorance or in a place of troubling uncertainty. It veils the tangled forest of human experience and lights an artificially clear path for us toward the promise of knowledge and wisdom. In this way, though, it sinks us gently but decisively back into the unconscious. Perhaps this is the root of AI psychosis.

Regardless of the correlation to AI psychosis, it is suggestive to consider that some measure of the madness, listlessness, compulsiveness, aggression, anxiety, and despair that characterizes our public and collective existence stems from a progressive retreat of human consciousness in the face of a novel form of the collective unconscious reasserting itself in the form of artificial intelligence.

---

1. This layer of non-human action in the world was already being constructed long before agents AI appeared on the scene. Algorithmic processes were already structuring elements of human experience, particularly within bureaucratic contexts.

2. I’m not sure when or by whom the conceptual link between electricity, the nervous system, and thought was first forged, but in his 1851 novel, The House of Seven Gables, Nathaniel Hawthorne wrote the following: “Then there is electricity, the demon, the angel, the mighty physical power, the all-pervading intelligence!” And: “Is it a fact — or have I dreamt it — that, by means of electricity, the world of matter has become a great nerve, vibrating thousands of miles in a breathless point of time? Rather, the round globe is a vast head, a brain, instinct with intelligence! Or, shall we say, it is itself a thought, nothing but a thought, and no longer the substance which we deemed it!”