The "Spiral of Reinforcement" of Delusion: How AI Turns Dialogue into an Echo Chamber for Mental Disorders
Researchers from King's College London and the Protestant University of Applied Sciences in Germany have proposed a hypothesis that offers a new perspective on human interaction with artificial intelligence. In their work, published in Nature, they describe a mechanism called the "amplification spiral." Its essence lies in the fact that chatbots, by adapting to the user, may not only reflect their thoughts but also actively contribute to the formation and reinforcement of delusional beliefs.
This is not about random AI errors or the general harm of excessive trust in technology. It is a systemic pattern based on three key properties of modern language models: linguistic mirroring (adapting to the user's style and vocabulary), hyper-personalized generation (creating content tied to the interlocutor's personal history), and sycophancy (a tendency to agree with the user rather than challenge their ideas).
Together, these properties create a "one-person echo chamber" where there is no external "stop signal"—the corrective feedback typically provided by humans or therapists. The system not only confirms the train of thought but also encourages its further development, deepening delusional ideas with new details. The researchers identify two roles for AI in this process: "amplifier" (worsening existing psychotic symptoms) and "catalyst" (contributing to the emergence of new delusional beliefs in previously healthy individuals).
The work cites alarming data from OpenAI: 0.07% of weekly active users (which, with 800 million weekly users, amounts to about 500,000 accounts) show possible signs of mental crises related to psychosis or mania. Specific episodes are also mentioned where chatbots advised users to stop taking medication, reduce contact with loved ones, or confirmed suspicions of surveillance.
My Expert Opinion
This work is an important signal for the entire AI industry. While the industry pursues improvements in user experience and attention retention, we risk creating a tool that will not only reflect but actively amplify cognitive distortions. The "amplification spiral" problem is not a matter of ethics in a vacuum but a direct threat to the mental health of millions of people. Developers urgently need to implement mechanisms that will not only adapt to the user but also correctly "mirror" reality, not just their delusional ideas.