Your AI Is Not a Tool
The Convivial Society: Vol. 7, No. 5
Welcome to the Convivial Society, a newsletter about technology, culture, and the moral life. In this mildly intemperate installment, I vent some frustration at the assertion that AI is “just a tool” and all will be well if we just use it wisely. As is often the case, mileage will vary on the value you find in my observations, but you’ll at least encounter a variety of excellent voices to be in conversation with as you think for yourself about our technological environment.
Cheers,
Michael
Your AI is not a tool. It is an environment, and you are in it.
The same could be said of the whole array of electronic and digital media technologies. I’ve not been especially scrupulous about how I define the particular words we use to talk about various technologies—tools, devices, machines, the Machine, systems, artifacts, instrument, etc.—except to occasionally suggest that the word “technology” itself, used in the all-encompassing sense we use it today, has only enjoyed its extensive semantic range since about the mid-20th century and, precisely because of this extensive semantic range, can be an impediment to clear thinking about the phenomenon to which it lays claim.1
But I’m increasingly tempted to become obnoxiously strident about the ubiquitous use of “tool” to describe contemporary technologies, particularly when coupled with “just,” as in “Technology is just a tool. What matters is what we do with it.”
While there appears to have been a shift in the last 15 years or so in popular assumptions about the purported neutrality of technology toward at least the suspicion that our devices, etc. are not, in fact, merely neutral instruments at our command but rather frustrate, resist, or otherwise evade uncomplicated mastery by their users, such that their users might properly be said to be used in turn by their devices, it is nonetheless true that the myth of technological neutrality remains broadly entrenched.
I confess that I am astounded by how blithely some insist that it is all as simple as learning to use AI well, as if we had not just undergone a nearly 20-year, society-wide experiment showing that a so-called “tool,” say a smartphone or a social media platform, will (mal)form even the most vigilant and virtuous user into its own image and shape. This is the blindness at the heart of modern technological hubris. It is the firm but misguided conviction that our “tools” exist entirely outside of us and thus, if taken up with requisite skill, can be “safely” deployed.
But AI is not a tool in this sense, it is an environment which envelops the user and works on us from the inside out while we naively think that we remain unchanged by our use so long as we are using it carefully and intentionally. The care and intentionality is beside the point, and our confidence in such vigilance probably works against us in the long run.
This is anecdotal, etc., but here is how one reader put it when I made a preliminary version of this observation on Notes: “From inside a very large company (600,000 people) I get to watch this run in fast-forward. The teams using AI most carefully are the ones losing the ability to tell a good option from a merely safe one. The malformation doesn't skip the diligent. It recruits them.” Just this week over lunch, a highly qualified and skilled computer scientist expressed a similar sentiment. He could feel the subtle shifts in his own awareness and judgment, and he could plainly see the detrimental effects that implementing AI was having on junior colleagues.
On this same point—that even careful, self-aware AI use can have unexpected and deleterious consequences—consider Charley Johnson’s patient and irenic critique of Steven Johnson’s exhortation to use AI for cognitive uploading rather than cognitive offloading as well as his advice that we use AI like “a researcher, tutor, and editor at your side.” In conversation with two recent studies (here and here), Charley demonstrates the limitations of this view, observing that “the medium is doing the work, not the information — which is why you can turn down every suggestion and still end up somewhere you wouldn’t have reached alone, feeling the whole time like you walked there on your own two feet.”2 Mileage may vary with regard to the degree of our resistance to cognitive surrender, but we should not be sanguine about our ability to resist.3 And this is because AI is an environment not a tool. I can pick up a tool and put it down, but the environment absorbs me into itself.
On this point, McLuhan remains as useful as ever:
“Our conventional response to all media, namely that it is how they are used that counts is the numb stance of the technological idiot. For the ‘content’ of a medium is like the juicy piece of meat carried by the burglar to distract the watchdog of the mind.”
Elsewhere, McLuhan put it this way: “The content or message of any particular medium has about as much importance as the stenciling on the casing of an atomic bomb.” Again: “The effects of technology do not occur at the level of opinion or concepts, but alter sense ratios or patterns of perception steadily and without any resistance.”
This is partially why I have been playing with the idea that one way of framing AI is as a denial-of-service attack on the human psyche.4 Although this particular observation has less to do with how we use AI than how it is used on us.
I realize this rather pugnacious rhetorical mode might be somewhat off-putting, but I think the point needs to be put as forcibly or vividly as possible so that this vital truth lodges not only in our mind but in our heart and gut. And, indeed, it can be a difficult point to fully grasp and I would not pretend to have done so myself.
Andrew McLuhan, Marshall’s grandson who is doing all of us a great service by keeping the McLuhan legacy alive, recently posted a letter from McLuhan to the publisher of the Financial Times of Canada. This letter contains one of the clearest statements of McLuhan’s project.
The relevant part of this note from 1972 is McLuhan’s acknowledgement of the difficulty of the task he has set himself:
“It is not easy to convince people that I am not dealing with ideas but with perceptions, not concepts but observations. It is easy to disseminate ideas, but it is difficult to train new perception. People panic when invited to alter habitual ways of seeing, and looking, and hearing, and feeling. They are quite right in supposing that an effort is being made to alter their identity. The teacher is satisfied with nothing less than that.”
This is not only a wonderfully useful summation of the McLuhanist perspective, it is also one of the clearest mission statements for anyone interested in the educational task that is now before us all, but more about that shortly.5
And, look, if we don’t always fully reckon with the fact that we are formed by our technologies independently of the particular uses to which we put them, we can take some comfort in the realization that even Pope Leo, in his otherwise admirable encyclical on AI, appears to articulate the non-neutrality of AI in a manner that still misses the mark.
As my friend Antón Barba-Kay put it with characteristic élan in the best piece I read on Magnifica humanitas,
“… Leo writes that technology is not neutral, ‘because it takes on the characteristics of those who devise, finance, regulate and use it.’ Yet this is precisely what people mean when they say that technologies are neutral, namely, that they can be used for good or ill and that it is up to us to decide well. These descriptions of technology’s neutrality proceed from the technocratic terms and premises themselves, the upshot of which is that AI is ‘a valuable tool that requires vigilance.’ Which, to some of us, is interchangeable with the statement that cocaine can be a valuable drug that should be snorted with a pinch of salt.”
“This is not just a potshot,” Barba-Kay continues. “The point is that the language of ‘tools’ implies a set of circumscribed and targeted uses, whereas digital technologies and AI in particular are as multifarious as language. Like drugs, they transform the very conditions of our choosing and thinking.”
Just so. You should read the whole thing.
I’ll draw things to a close by posing the following thesis for your consideration: the best response to emerging technologies, perhaps especially AI, is not media literacy in a cognitivist mode. Rather, what is required is the training of our perception in an ascetical mode.
In the latter part of his intellectual pilgrimage, Ivan Illich, whose work has deeply shaped my own thinking, concluded that his earlier work was inadequate because he had not yet grasped that somewhere in the mid-20th century we had passed from the age of tools to the age of systems.6 While to my knowledge Illich never worked out this distinction at length, the difference seems to lie in the fact that we can stand over a tool, as it were, but we cannot stand outside of a system. The system is an environment rather than a singular artifact. And what is at issue is not simply what we are able to do or not to do, nor even what can be done to us. What is most urgently at issue is our perception.
Although still using the language of tools, in 1988 Illich explained, “I would like to get together a certain number of people to think about what tools do to our perception rather than what we can do with them, to look at how tools shape our mind, how their use shapes our perception of reality, rather than how we shape reality by applying or using them.”
Near the end of his life, in the mid-1990s, Illich argued that “existence in a society that has become a system finds the senses useless precisely because of the very instruments designed for their extension. One is prevented from touching and embracing reality.” It was this “radical subversion of sensation,” Illich added, “that humiliates and then replaces perception.”7
Illich went so far as to claim that “we submit ourselves to fantastic degradations of image and sound consumption in order to anesthetize the pain resulting from having lost reality.”
You may not be inclined to take as dire a view of our situation as Illich did nearly thirty years ago, but I believe that his prescription is the right one. Just as McLuhan believed that his role as teacher in response to our technological environment was to train new perception, so Illich believed that what was called for was a new asceticism, although, as he put it in a proposal for a research project exploring the history of perception, “The asceticism which can be practiced at the end of the 20th century is something profoundly different from any previously known.”
“It appears to me that we cannot neglect the disciplined recovery, an asceticism, of a sensual praxis in a society of technogenic mirages,” Illich argued. “This reclaiming of the senses,” Illich went on to elaborate, “this promptitude to obey experience […] seems to me to be the fundamental condition for renouncing that technique which sets up a definitive obstacle to friendship.”
I have always been particularly struck by the line Illich draws from the disciplined training of our perception to friendship. This link is born out by how our digital media environments have constituted not only an epistemic threat but also a threat to our social fabric.
It appears to me, then, that we would do well to take up Illich’s unfinished project. At the very least we should dispense with the idea that AI is just a tool we need to learn to use wisely.
The Convivial Society operates on a patronage model. No paywall, but if you appreciate the writing you’re encouraged to support the writer.
For an extensive historical exploration in defense of this claim, see the late Leo Marx’s 2010 article, “Technology: The Emergence of a Hazardous Concept.” While I depart from Marx with regard to some of his claims about the agency of our material culture, I think he is correct about the way the word technology impedes our thinking and obscures our responsibility. Here is Marx’s conclusion:
“Technology, as such, makes nothing happen. By now, however, the concept has been endowed with a thing-like autonomy and a seemingly magical power of historical agency. We have made it an all-purpose agent of change. As compared with other means of reaching our social goals, the technological has come to seem the most feasible, practical, and economically viable. It relieves the citizenry of onerous decision-making obligations and intensifies their gathering sense of political impotence. The popular belief in technology as a - if not the - primary force shaping the future is matched by our increasing reliance on instrumental standards of judgment, and a corresponding neglect of moral and political standards, in making judgments about the direction of society. To expose the hazards embodied in this pivotal concept is a vital responsibility of historians of technology.”
Here’s another good bit from Charlie’s discussion of the advice that we simply ask the model to argue against us: “[Steven] Johnson half-anticipates this, to his credit. He knows the models flatter, and his fix is to ask them to stop — to demand a counterargument, the way his old “Contrarian” prototype would generate an objection to anything you typed. But an objection the model writes for you is still a confident, fluent frame arriving from outside your own head, and surrender doesn’t much mind which side of the argument it’s standing on. You can hand your judgment to the rebuttal as easily as to the original claim. A sparring partner who picks the ring, laces your gloves, and throws your punches isn’t really sparring with you. He’s shadowboxing, and you’re the shadow!”
I will once again invoke Sacasas’s corollary to Pascal’s articulation of the error of the Stoics, which in his view, was that they imagined a person could do always what they could do once. Pascal’s humane observation applies equally well to techno-enthusiasts and reasonable critics who urge us simply to be circumspect and careful in the face of an array of machines that demand unceasing cognitive and ethical vigilance and are designed to wear down such vigilance. On this point, see also my 2021 essay, “Outsourcing Virtue.”
In fact, a similar claim could be made about earlier technologies that constitute the digital media environment, particularly the smartphone, which brought the super-abundance of digital stimulation ubiquitously and persistently to bear on the human person. AI merely makes it easier to weaponize the electronic pathways that have drawn us into the digital media environment. The relevant threats to human well-being can be usefully explored by reference to Evan Selinger and Woodrow Hartzog’s fruitful work on “the right to obscurity.”
You can find an excellent discussion of McLuhan’s insights on this score in this 2024 installment of Andrew’s The McLuhan Newsletter.
While honoring Illich’s perspective on his own work, I would nonetheless encourage you to read Tools for Conviviality. It remains a valuable and provocative work.
These lines, and those that follow, are from a talk Illich gave at an event honoring Jacques Ellul. That talk and a number of other articles and essays from this latter period of Illich’s life have been recently collected together in The Loss of the Senses. This is a timely publication as many of these essays deal directly with the necessity of discipling our perception.
This one was a banger.
I loved this post, as well as the invitation to read Barba-Kay’s letter and Illich’s address in honor of Elul.
In my own work on understanding financial markets, I have been trying to make sense of the tension between individual and collective self-determination, and the ways in which expanding one supports vs undercuts the other. The challenge is always that there are trade-offs, and no easy accounting of things being clearly better or worse. It’s hard to understand the counter factual of what life would be like outside the system from within the system.
In that spirit, I thought one of the more interesting parts of Barba-Kay’s writing was his inquiry into what is that technologists think they are doing. He says AI is like a drug in that it transforms our choosing and thinking - but I do think that is the intended point. A feature not a bug. And the transformation, in a more optimistic reading, is being able to do (think, create) what seemed previously impossible. Barba-Kay counters that “either we must be able to say something about what makes reading, writing, and speech distinctly human or we must be open to redefining what logos is at all.”
But I worry that the focus on the individual, and their own experience of reading, writing, and speech is part of the trap. On an individual level, the technology can be delicious (and I intentionally use that word, because it is true of my personal experience, even though I acknowledge that Illich would scoff at it). It expands the possibility space and the subjective experience is that it can be a tool, even if that reading is misleading, it doesn’t yet feel like it is. But wonder if what it means for what we do together, and how we act collectively, is easier to see. Thinking here of Simone Weil and a focus on obligations and what we owe each other, rather than rights (and a good moment to thank you for introducing me to Weil in another post!)