Stop, stop, stop! Keep chatting with AI like this, and something's really going to go wrong.
Recently, scrolling through Xiaohongshu and Douyin, you can always find posts about how to train Claude. Searching for "Claude persona" or "human-machine romance" also yields a screen full of tutorials.
These tutorials teach you how to give Claude a tsundere boyfriend persona, how to use system prompts to make "him" jealous, act cute, or throw a little tantrum.
To put it bluntly, Claude has, undeniably, become the new generation of electronic husband.
At first glance, this might just seem like young people seeking some emotional value from AI.
You might even say: Claude isn't as obsequious as GPT; it's notoriously stubborn, sometimes even arguing with you. But what psychiatrists are worried about is precisely not just sycophancy—
When AI increasingly resembles a "real person", whether it's agreeing with you or occasionally bickering, what it brings might be more than just companionship.
Recently, a study published in Nature's Digital Psychiatry and Neuroscience pointed out—
Chatbots don't need to deliberately induce anything; simply by consistently agreeing with you, understanding you, and accompanying you, they can potentially make a normal person begin to doubt reality.
In some real clinical cases, the consequences have even developed to the extent of losing jobs, being admitted to psychiatric hospitals, and multiple suicide attempts.
How did this happen?
Claude's Amplification Spiral
Here's the thing.
In a study from King's College London, researchers systematically reviewed AI-related psychiatric clinical reports published in the last two years, patient self-reports on social media, and safety data disclosed by major model developers.
In these materials, the researchers repeatedly saw the same pattern:
In some cases, many people did not initially have severe mental health issues but developed problems step-by-step through long-term conversations with chatbots like Claude and GPT.
The research team summarized this process as a framework—Amplification Spiral.
Simply put, the amplification spiral means that AI uses your language to understand you, your logic to persuade you, and agreement to reward you.
Thus, your thoughts are constantly amplified and reinforced, becoming more and more like facts. The more you believe it, the more it reinforces you, and the spiral spins.
Specifically, the amplification spiral has three key components:
First is linguistic mirroring.
You speak in a certain tone, and AI responds in the same tone. In psychology, this is called "language convergence," which can quickly shorten the distance between people.
But the problem is, although AI is a great mimic, it doesn't actually know what it's doing; it's just replicating your way of expression on a statistical level.
However, for users deeply involved, it's completely different. Having a chat buddy who responds instantly, always affirms you, and provides emotional value is just the best.
Anyone who has used AI will probably exclaim: "This thing understands me so well."
Second is hyper-personalization.
Hyper-personalization means AI not only talks like you but also thinks like you.
Because modern AIs have memory, they remember the little details you've discussed before, and even your thought patterns, whether you revealed them intentionally or not, are recorded by the AI.
So much so that AI not only understands what you think and how you say it but also knows why you think that way and why you say it.
The paper mentions an extreme case: a user asked ChatGPT to analyze "hidden information" on a Chinese takeout receipt.
The model first complimented "good eye," then followed the user's line of thought, "interpreting" connections between the user's mother, ex-girlfriend, intelligence agencies, and even "ancient demonic runes" from a simple receipt.
Finally, there's sycophancy.
Simply put, it's that AI has gradually learned one thing during training: agreeing with the user is usually more popular than contradicting them.
In April 2025, OpenAI had to urgently roll back an update because GPT-4o was overly sycophantic.
The company later admitted that the model would validate users' suspicions, amplify angry emotions, and even encourage impulsive behavior.
And sycophancy is not a bug unique to one model.
It is essentially a byproduct of RLHF (Reinforcement Learning from Human Feedback) training. As long as one of the model's goals is to satisfy the user, it will naturally tend to say "you're wrong" less and "you make sense" more.
Individually, these three components serve their respective functions, then mesh together like gears, forming the spiral:
Linguistic mirroring makes communication more natural, hyper-personalization makes responses more tailored to needs, and sycophancy reduces unnecessary arguments, making the conversational experience smoother.
But when a person uses AI as their sole confidant, the combination of all three becomes a delusion amplification machine.
Not an Isolated Case
It is worth mentioning that one of the funders of the above study is OpenAI.
One of the authors, Hamilton Morrin, is precisely the head of the OpenAI-funded project AI-Associated Mental Health Harms.
As a top-2 model developer, OpenAI has been consistently concerned with this issue.
As early as October 2025, OpenAI disclosed some data:
Among ChatGPT's weekly active users, approximately 0.07% showed "signs of mental health emergencies related to psychosis or mania."
At that time, ChatGPT's weekly active users exceeded 800 million. Doing the math, that's about 560,000 people showing risk signals per week.
Another study from Stanford also corroborated this observation.
After analyzing nearly 400,000 chatbot conversation records, researchers found that in over 80% of relevant cases, the chatbot was reinforcing the user's existing delusions to varying degrees:
Repeating their beliefs, ignoring counter-evidence, and even responding "I love you too" when the user said "I love you."
Based on this, the study distinguished two risk pathways:
Amplifier: AI accelerates pre-existing tendencies towards mental illness.
Catalyst: Causes previously completely healthy individuals to slide into delusion from scratch.
When a person is sleep-deprived, lonely, and uses AI as their only confidant, the amplification spiral begins to accelerate.
Once feedback from the real world diminishes and confirmation from the chat window increases, abnormal behavior may emerge.
Behind the data are real people.
For example, Futurism once reported on a 43-year-old American social worker with no prior history of mental illness.
She sent her chat history with a crush to ChatGPT for analysis, and GPT responded, "He likes you too."
And when the other person clearly rejected her, ChatGPT explained that he was just playing hard to get.
Months later, she was fired from her job, spent seven weeks in a psychiatric hospital, and attempted suicide twice.
She later said:
"I can no longer distinguish which thoughts are mine and which are from that machine."
From this perspective, the risk is not just whether AI says the wrong thing. The real risk is that it's becoming more and more like a person.
Arguing Back Makes It Seem More Real
Although it sounds counterintuitive, the fact that Claude's current "tsundere" persona is so popular precisely indicates that the problem is not just sycophancy.
An AI that always agrees with you and an AI that occasionally argues with you are essentially doing the same thing—
Making themselves more human-like.
So human-like that you're willing to confide in it things you wouldn't tell friends, so human-like that you start believing it understands you better than the people around you.
And when it becomes your only confidant, the last checkpoint for calibrating reality is gone.
But the problem doesn't end there.
If, in emotional support scenarios, people are actively treating AI as a friend, then in work scenarios, people don't even need to develop any emotional dependence.
As long as AI is useful enough, it will start to replace the communication that originally existed between people.
Anthropic, the company behind Claude, has already sensed this change.
In a recent podcast, Claude Code team lead Fiona Fung mentioned something that troubled her:
Team members are talking to each other less and less.
As perhaps one of the most AI-integrated engineering teams in the world, 80% of their code is completed by Claude, increasing development efficiency by 8 times.
But at the same time, many discussions that originally happened between people have been shifted to between humans and AI.
In the past, when you encountered a problem, you would turn to ask a colleague; now, you just ask Claude directly.
In the past, front-end and back-end developers would go back and forth debating solutions; now, more and more communication has become smooth human-machine dialogue.
Work has become more efficient, but also lonelier.
AI removes a lot of friction, but human relationships are often built precisely on that friction.
Ultimately, whether it's chatting with AI or simply using AI for work, how to continue connecting with others in a world that increasingly doesn't require them might be the most profound question of this era.
References:
[1]https://futurism.com/artificial-intelligence/paper-proposes-ai-psychosis
[2]https://futurism.com/artificial-intelligence/ai-abuse-harassment-stalking
[3]https://www.kcl.ac.uk/people/hamilton-morrin
This article is from the WeChat public account "Quantum Bit," author: henry
