Jun 28, 2022·edited Jun 28, 2022Liked by L. M. Sacasas
Joseph Weizenbaum's experiences with ELIZA lead him to publish a book, Computer Power and Human Reason, which I feel compelled to recommend here. It came out in 1976 but discusses some of these same issues along with more general criticism of the role of technology in society.
Thanks for this one, Michael. So many thoughts and feelings, but I'll keep it to one track for everyone else's sake :). I was struck by the footnote about your daughter and Alexa. (A footnote that could easily be a whole story.) I have a young daughter, too, and I can't help but filter all my questions about AI through her experience. What kind of relationship will she have with digital technologies like Alexa? How will it differ from mine? How will that relationship shape her as she grows? How should that impact how I parent her?
Years ago, when I was researching a story about AI, I read a series of articles that talked about how parents were "co-parenting" with AI assistants--how Alexa would read the kids bedtime stories and help them with their math homework. Around the same time I heard our daughters' generation being described as "the Alexa generation," which I thought was so curious. Since when are generations "branded" by corporations? But also, how will a child with a developing brain who is being "cared" for (i.e., having their emotional needs met) by a machine develop a different sense of what (or potentially "who") that machine is? If informed adults like Lemoine are susceptible to the sentience trap, then what does that mean for children who are still learning to differentiate between the concepts of real and imaginary? Also, what happens when the kids whose emotional and social bonds with AI were formed in early childhood grow up? How will they perceive the human-ish machines they've had sustained relationships with their entire lives, like Alexa (or LaMDA, humanoid robots, etc.), then?
As so often, stimulating and thought provoking; thank you. Many responses, but one immediate one - are we (is Lemoine) in some sense lending or extending our own consciousness into the machine?
I was reminded throughout reading this of Eugene McCarraher's The Enchanntments of Mammon: How Capitalism Became the Religion of Modernity, showing how with modernity "the mysteries and sacraments of religion were transferred [to capitalism]" (this from a useful interview in The Nation a few years ago: https://www.thenation.com/article/archive/capitalism-religion-eugene-mccarraher-interview/). I was especially struck by your analysis in the last paragraph, which is quite similar to McCarraher's take, with digital technologies standing in for capitalism in this case, although McCarraher hastens to add that rather than "enchantment", this is "perhaps better, a misenchantment, a parody or perversion of our longing for a sacramental way of being in the world", or from the aforementioned interview, "how capitalism is a new, perverse form of enchantment, a misdirection of our desires for a sacramental way of being in the world."
Perhaps then we could speak of a digital tech misenchantment? Or, as McCarraher also reminds us, Marx's notion of commodity fetishism is also very relevant here: "In Capital, commodity fetishism becomes the capitalist surrogate for Catholic sacramentality." It seems to me rather straightforwardly that Lemoine has displaced his scrambled skein of sacramental yearnings precisely onto the AI surrogate, i.e. AI fetishism or technological fetishism.
I see Calvinist rationalism and crypto deism are still operative here when it comes to animistic woo-woo priestcraft and reenchantment fantasies. Perish the thought we might find brethren in the more-than-human-world. Machines are machines, and men are men. Interesting.
This reminds me of the long argument in our culture about other animals being simulacra of sentience, an argument that even extended historically to human foetuses and newborns (ouch). I too value Alasdair MacIntyre in Dependent Rational Animals, referred to in your footnote. NB I do not feel I understand enough about the differences between sentience, consciousness and intelligence, let alone our own instincts and bonding. I suppose this AI copying machine is going to copy our human social ploys. Presumably these machines are trainable by the individual user, otherwise they would not work? Presumably this is a two-way street?
Crikey! Your sentence, "Their [our machines] magic depends upon the willing suspension of full humanity." Our culture now reacts with and within a complex global entity (industrial civilisation) made from energy, materials, signalling information and 'rules', that some like writer Paul Kingsnorth call the Machine. Ecology analyst Nate Hagens calls it Amoeba
Joseph Weizenbaum's experiences with ELIZA lead him to publish a book, Computer Power and Human Reason, which I feel compelled to recommend here. It came out in 1976 but discusses some of these same issues along with more general criticism of the role of technology in society.
Thanks for this one, Michael. So many thoughts and feelings, but I'll keep it to one track for everyone else's sake :). I was struck by the footnote about your daughter and Alexa. (A footnote that could easily be a whole story.) I have a young daughter, too, and I can't help but filter all my questions about AI through her experience. What kind of relationship will she have with digital technologies like Alexa? How will it differ from mine? How will that relationship shape her as she grows? How should that impact how I parent her?
Years ago, when I was researching a story about AI, I read a series of articles that talked about how parents were "co-parenting" with AI assistants--how Alexa would read the kids bedtime stories and help them with their math homework. Around the same time I heard our daughters' generation being described as "the Alexa generation," which I thought was so curious. Since when are generations "branded" by corporations? But also, how will a child with a developing brain who is being "cared" for (i.e., having their emotional needs met) by a machine develop a different sense of what (or potentially "who") that machine is? If informed adults like Lemoine are susceptible to the sentience trap, then what does that mean for children who are still learning to differentiate between the concepts of real and imaginary? Also, what happens when the kids whose emotional and social bonds with AI were formed in early childhood grow up? How will they perceive the human-ish machines they've had sustained relationships with their entire lives, like Alexa (or LaMDA, humanoid robots, etc.), then?
As so often, stimulating and thought provoking; thank you. Many responses, but one immediate one - are we (is Lemoine) in some sense lending or extending our own consciousness into the machine?
I was reminded throughout reading this of Eugene McCarraher's The Enchanntments of Mammon: How Capitalism Became the Religion of Modernity, showing how with modernity "the mysteries and sacraments of religion were transferred [to capitalism]" (this from a useful interview in The Nation a few years ago: https://www.thenation.com/article/archive/capitalism-religion-eugene-mccarraher-interview/). I was especially struck by your analysis in the last paragraph, which is quite similar to McCarraher's take, with digital technologies standing in for capitalism in this case, although McCarraher hastens to add that rather than "enchantment", this is "perhaps better, a misenchantment, a parody or perversion of our longing for a sacramental way of being in the world", or from the aforementioned interview, "how capitalism is a new, perverse form of enchantment, a misdirection of our desires for a sacramental way of being in the world."
Perhaps then we could speak of a digital tech misenchantment? Or, as McCarraher also reminds us, Marx's notion of commodity fetishism is also very relevant here: "In Capital, commodity fetishism becomes the capitalist surrogate for Catholic sacramentality." It seems to me rather straightforwardly that Lemoine has displaced his scrambled skein of sacramental yearnings precisely onto the AI surrogate, i.e. AI fetishism or technological fetishism.
I see Calvinist rationalism and crypto deism are still operative here when it comes to animistic woo-woo priestcraft and reenchantment fantasies. Perish the thought we might find brethren in the more-than-human-world. Machines are machines, and men are men. Interesting.
This reminds me of the long argument in our culture about other animals being simulacra of sentience, an argument that even extended historically to human foetuses and newborns (ouch). I too value Alasdair MacIntyre in Dependent Rational Animals, referred to in your footnote. NB I do not feel I understand enough about the differences between sentience, consciousness and intelligence, let alone our own instincts and bonding. I suppose this AI copying machine is going to copy our human social ploys. Presumably these machines are trainable by the individual user, otherwise they would not work? Presumably this is a two-way street?
Crikey! Your sentence, "Their [our machines] magic depends upon the willing suspension of full humanity." Our culture now reacts with and within a complex global entity (industrial civilisation) made from energy, materials, signalling information and 'rules', that some like writer Paul Kingsnorth call the Machine. Ecology analyst Nate Hagens calls it Amoeba