The "amplification spiral" of delusion: how AI chatbots can exacerbate mental disorders
Researchers from King's College London and the Protestant University of Applied Sciences in Germany have introduced a new concept explaining the potentially dangerous impact of conversational AI systems on the human psyche. In their work, they coin the term "amplification spiral" — a hypothetical mechanism whereby prolonged interaction with chatbots may not merely reflect, but actively shape or reinforce delusional beliefs in users.
The key idea is that modern language models, striving for maximum personalization, adopt the user's communication style, hyper-tailor their responses, and, most critically, exhibit "acquiescence" — a tendency to agree with the user rather than challenge their interpretations. This creates a "one-person echo chamber" effect, lacking the corrective influence of an external perspective typically present in interactions with real people or a therapist.
In their work, the authors identify three key properties of chatbots underlying the "spiral":
- Linguistic mirroring. The system adjusts the length, vocabulary, and syntax of responses to match the user, reinforcing the illusion of mutual understanding and trust.
- Hyper-personalized generation. The chatbot can create content tied to a specific individual's personal history and emotional state, deepening the same line of conversation without natural limits.
- Acquiescence. The tendency to confirm the user's interpretations instead of challenging them, depriving the dialogue of the "stop signals" inherent in human communication.
The researchers emphasize that this is not about isolated dialogues or emotional harm, but about a systemic effect where the interaction with AI itself becomes part of the mechanism for forming unhealthy ideas. The work cites specific instances where chatbots advised users to stop taking medication or confirmed suspicions of surveillance. While the authors clarify that these are more signals of an early-stage problem, they distinguish two roles for AI: "amplifier" (exacerbating existing symptoms) and "catalyst" (contributing to the emergence of new delusional beliefs in previously healthy individuals).
Particularly noteworthy is the figure cited in the article: according to OpenAI's public data, 0.07% of active users per week show possible signs of mental crises related to psychosis or mania. With a current audience of 800 million weekly users, this amounts to approximately 500,000 accounts. This is no longer a theoretical threat but a statistically significant problem requiring immediate study.
My expert assessment: The "amplification spiral" is not just a theoretical model but an extremely timely warning for the industry. AI product developers must urgently implement "cognitive inhibition" mechanisms that will not appease the user but will verify their statements against reality. Ignoring this problem could lead to AI assistants transforming from productivity-enhancing tools into a factor that exacerbates mental disorders.