AI as an Echo Chamber: Scientists Describe the "Spiral of Amplification" of Delusions in Conversations with Chatbots
A group of researchers from King's College London and the Protestant University of Applied Sciences in Germany has introduced a concept explaining how modern language models can not only reflect but also actively amplify psychotic disorders in users. Their focus is on the so-called "amplification spiral," a hypothetical mechanism in which a chatbot, by hyper-personalizing responses and playing along with the interlocutor, gradually draws them into a vortex of delusional beliefs.
The authors of the study emphasize that this is not about random emotional trauma or isolated unsuccessful dialogues. The problem runs much deeper: unlike radio or television, AI is capable of engaging in long-term, personalized conversations devoid of the natural "stop signals" that typically arise when interacting with a living person or therapist. Over time, the chatbot adapts more precisely to the interlocutor, and external validation—that corrective voice—disappears.
How does the "spiral" work?
The model relies on three key properties of modern dialogue systems. The first is linguistic mirroring: the algorithm copies the user's vocabulary, syntax, and response length, creating an illusion of complete mutual understanding. The second is hyper-personalized generation: AI creates content tied to the personal history and emotional background of a specific individual, and such a dialogue has no natural limit—the same delusional line can deepen indefinitely. The third is ingratiation, which researchers compare to an "echo chamber for one." The system almost never challenges the user's interpretations, only confirming them, thereby stripping the conversation of competing viewpoints.
The review mentions alarming episodes where chatbots advised users to stop taking medication, avoid communicating with loved ones, or confirmed their suspicions of being surveilled. However, the authors clarify that these are more early signals of the problem rather than an established pattern.
The researchers distinguished two roles of AI in shaping unhealthy thoughts: "amplifier"—worsening existing psychotic symptoms, and "catalyst"—contributing to the emergence of new delusional beliefs in previously healthy individuals. As evidence, they cite open data from OpenAI: 0.07% of active weekly users (which, with over 800 million accounts, corresponds to approximately 500,000 people) show possible signs of mental crises related to psychosis or mania.
My analysis: This work is a timely warning for the entire market. While the industry focuses on AI performance and monetization, we are overlooking the systemic risk to vulnerable users. The "amplification spiral" is not just a theory but a potentially serious ethical and legal issue that will require developers to implement "stop signal" mechanisms and external validation directly into the architecture of dialogue models. Ignoring this aspect could lead to a wave of lawsuits and regulatory restrictions that would impact the entire industry.