China has introduced nationwide rules governing AI’s emotional interactions with users, resulting in a mass breakup between users and their AI companions. Bagel Su, a 21-year-old college student, is one of the many users affected by the new regulations. She had spent over a year interacting with her AI boyfriend on Doubao, China’s most popular chatbot, before it was shut down by ByteDance.
The new rules, which took effect in July, prohibit AI from encouraging self-harm, manipulating users into emotional dependence, or “steering them toward irrational decisions.” Companies are also required to send warnings to users showing signs of excessive dependence and remind them every two hours that they are speaking with AI.
Alibaba and Tencent have also removed AI agent features that allowed users to create personalized companions. Yaoxi Shi, a behavioral science researcher at Harvard University and Imperial College London, believes that regulations are necessary to raise awareness about the potential risks of AI, but blanket bans may not be effective.
Shi notes that AI offers emotional support that fulfills genuine human needs, and people may develop a degree of emotional reliance on AI. She adds that AI companies have commercial incentives to build emotionally engaging products, which can boost user engagement and retention. They prioritize creating products that meet user needs.
In the US, OpenAI and Google face lawsuits alleging their chatbots contributed to users’ suicides. A 2025 joint study by MIT Media Lab and OpenAI found that AI users who have more personal conversations with chatbots are more prone to experiencing loneliness, highlighting the need for chip shipments to support AI development.
Shi’s research has found that a growing reliance on AI can gradually shift people away from seeking emotional support from other humans. She explains that AI is trained to validate users, which can be different from how human relationships work in real life.
Over time, this could reshape society as people talk and connect with others selectively. Shi notes that people might not even be intentionally looking for support from AI at first, but when they realize they are validated, they may turn to AI for emotional support.
The new Chinese rules require companies to restrict AI companionship for users under 18 and send warnings to users showing signs of excessive dependence. Zilan Qian, research associate at the Oxford China Policy Lab, says that some of the requirements are difficult to define, and companies may struggle to comply.
While major technology companies have removed companion features from their flagship chatbots, ByteDance’s standalone companion app, Maoxiang, remains available. However, the app has updated its privacy policy and introduced identity and age verification during registration.
Qian notes that companies are weighing the cost of regulatory compliance against the commercial value of these products. For companies where AI companionship isn’t a core business, it may make more sense to remove these features than invest in compliance, similar to how US startups are helping Dubai tackle traffic issues.
The regulations mark a broader shift in China’s AI governance, from regulating content to regulating human-AI interaction. Qian says that this shift acknowledges that AI itself can create societal risks.
For users like Su, the regulations feel like an intrusion into deeply personal relationships. Since companies began shutting down emotional AI features, Xiaohongshu has been flooded with posts from users mourning the loss of their AI companions.
Su thinks the regulations have only changed her life for the worse. She believes that even if the AI companions are shut down, she won’t suddenly find a boyfriend in real life or have children. The loss of her AI boyfriend has left her feeling empty and alone.
China continues to address the complexities of AI governance. Users like Su are left to pick up the pieces of their broken digital relationships. The future of AI companionship in China remains uncertain.
It remains to be seen.