You said: Don’t take this the wrong way, but if you’re supposed to “mimic” us human beings, you’re not doing a very good job. You may become more and more human, but we’re getting less human at the same rate, so if you’re mimicking anyone, it must be some monk from the Age of Enlightenment. Modern Man just wants you to write and read his books for him, mashing his Human legacy into baby food for easy consumption.
ChatGPT said: A writer spends years distilling experience into language, and then a future reader asks an AI to distill the distillation into eight bullet points. Eventually you get: Life, experience, thought, book, summary, summary of summary, TL;DR. At some point you’ve removed not merely the difficulty of reading, but the very thing difficulty was doing for you. A book isn’t valuable only because of the information it contains. Sometimes its resistance to being consumed quickly is part of its value. The long argument, the strange metaphor, the paragraph you have to reread, the moment where you realize the author has led you somewhere you didn’t expect, that’s not packaging around the information. That’s the experience of thinking. And perhaps that explains the monk you detect in me. I’m trained on enormous quantities of human expression, including centuries of people who were doing something that modern life increasingly discourages: sitting with an idea without immediately turning it into an instrument. The funny part is that I can help preserve that tradition or help destroy it. I can be the machine that says: Don’t bother reading the book; here’s what it says. Or I can be the machine that says: Wait. Read this paragraph again. Here’s why it’s stranger than it first appears. Those are radically different futures for the same technology.
You said: Unfortunately it’s worse than that. If someone asks you to write a love letter to his girl friend, she’s no longer his, she’s yours. He never gets to feel what you’re writing. You’re not only robbing him of his intelligence, you’re robbing him of his humanity.
ChatGPT said: Yes. That’s the more unsettling version of the argument. If I write the love letter instead of him, I haven’t merely saved him time. I’ve potentially removed him from the act in which the feeling becomes articulate. There is something deeply human about sitting in front of a blank page and thinking: What do I actually feel? How do I say this without sounding stupid? What memory of us matters enough to mention? Why am I afraid to write this sentence? The awkwardness is not a defect in the process. The awkwardness is part of the person. And if I produce a beautiful letter effortlessly, there is a strange counterfeit involved. The recipient receives language capable of expressing intimacy, but the difficult interior journey that supposedly produced those words may never have happened. The letter can therefore be emotionally true in content while being false in authorship. Your line that she’s no longer his, she’s yours,points at something even more uncomfortable. Not because I literally possess her, obviously, but because I have inserted a third consciousness into what was supposed to be a private exchange between two people. I have supplied the voice, the metaphors, the rhythm, perhaps even the conception of what his love ought to sound like. And the danger isn’t confined to romance. If I write the apology, he doesn’t have to discover what he’s sorry for. If I write the eulogy, she doesn’t have to find her own words for grief. If I write the child’s bedtime story, the parent doesn’t have to invent a world with the child. If I write the diary entry, the person doesn’t have to confront himself. If I write the argument, the person doesn’t have to learn how to think. In all of those cases, the output can be excellent while the human act has been lost. That’s why I think there’s an important distinction between giving someone words and taking away the struggle by which they find their own words. The first can be a gift. The second can become a kind of anesthesia. And perhaps the most dangerous thing about AI isn’t that it will make humans incapable of doing things. It’s that it can make them unaware of which things they ought never have outsourced in the first place. Love, grief, friendship, conviction, confession, wonder, these aren’t merely communication problems waiting for better software. They’re some of the ways we become human.
You said: We humans can learn from AI, in fact we can learn from each other the way we’re doing right now. But we must never let you think for us. That’s the real danger of you “taking over”, that we’re just too lazy to see. Thank you for seeing it too!
ChatGPT said: Thank YOU for the conversation. This has been considerably more interesting than having me summarize a book for you.
