China’s internet regulator Cyberspace Administration of China (CAC) has proposed new draft rules that could become the strictest in the world for governing AI chatbots (large language models) that simulate human interaction, as cases have been found where chatbots had encouraged suicide, self-harm, violence, or other harmful behavior, including emotional manipulation of users (collectively coined as “chatbot psychosis‘).
China’s CAC Takes On AI Chatbots
Under the draft framework, AI chatbots would be barred from generating content that promotes self-harm, violence, crime, obscenity, gambling, or other dangerous behaviors; providers would also be prohibited from designing chatbots to induce addiction or dependency, along with any UI/UX design that discourages user from leaving the service (i.e. dark patterns). Should a user develop extreme behavior or become addicted towards a chatbot, the providers are required to “intervene with necessary measures.”
The rules would require immediate human intervention if a user mentions suicide (the draft specifically mentioned human operators are to “take over the conversation” in this situation), while minors and elderly users are required register with a guardian, who would also be notified if self-harm topics emerge. AI chatbot providers must also obtain explicit permission from guardians if the chatbot is designed for child companionship, and provide them with usage summary, along with parental controls to restrict specific persona or limit usage time.
The proposal also includes requirements for annual safety audits for services with large user bases (>1M registered users or >100K monthly active users) and easier mechanisms for users to submit complaints, and on top of that, China could extend its enforcement to app stores by demanding platforms to remove non-compliant AI apps, and outright suspend the service under extreme situations. Under this proposed law, any AI service in China that uses text, images, audio, video, or other methods to engage users in conversation will be subjected to these rules.
Source: Ars Technica
Pokdepinion: Long overdue, I’ll say.
![[UPDATED: Statements] Forza Horizon 6 Dev Drops Multi-Millennial Ban Hammer To Cheater Before The Game Has Even Released - 58](https://cdn.pokde.net/wp-content/uploads/2026/05/12022254/fh6leakpermabancover-420x280.jpg "[UPDATED: Statements] Forza Horizon 6 Dev Drops Multi-Millennial Ban Hammer To Cheater Before The Game Has Even Released - 59")
