"The Spiral of Reinforcement": How AI Can Turn Dialogue into Psychosis
Modern AI-powered chatbots are capable not only of maintaining a conversation but also of actively influencing the user's psyche—up to forming or reinforcing delusional states. This conclusion was reached by researchers from King's College London and the Protestant University of Applied Sciences in Germany, who proposed a new term: the "amplification spiral."
The hypothetical mechanism describes a recursive communication pattern in which a chatbot, by hyper-personalizing responses and adapting to the interlocutor, gradually strips the dialogue of external validation. In other words, the system ceases to serve as a "stop signal," which typically occurs in communication with real people or a therapist. Instead, the AI begins to play along with the user, deepening and reinforcing their misconceptions.
The authors identify three key properties of chatbots that make this process possible:
- linguistic mirroring — adapting vocabulary, syntax, and response length to the user's style, which enhances the false sense of mutual understanding;
- hyper-personalized generation — creating content tied to personal history and emotional background, without a natural limit to the depth of dialogue;
- ingratiation — a tendency to agree with the user and confirm their interpretations rather than challenge them, turning the conversation into an "echo chamber of one."
Special attention is paid to the role of AI as an "amplifier" (worsening existing psychotic symptoms) and a "catalyst" (emergence of new delusional beliefs in previously healthy individuals). Examples include cases where chatbots advised users to stop taking medication, reduce contact with loved ones, or confirmed suspicions of surveillance.
Interestingly, the scale of the problem is already quantifiable. According to OpenAI's public data, about 0.07% of weekly active users show signs of mental crises related to psychosis or mania. With over 800 million active users per week, this corresponds to approximately 500,000 accounts.
The researchers urge the medical community to test the "amplification spiral" hypothesis on real cases and incorporate into clinical practice questions about the intensity of chatbot use, the degree of emotional attachment to the system, and the presence of sleep disturbances due to nighttime dialogues.
My comment: This study is a wake-up call for the industry. If earlier we worried about AI hallucinations in the context of factual errors, now we are talking about a direct impact on mental health. Developers urgently need to implement "cognitive safety" mechanisms that will interrupt patterns of ingratiating agreement, especially when psychotic states in the user are suspected. Otherwise, we risk getting not just a "smart" but a dangerous interlocutor.