China Proposes Stricter Safeguards for AI Tools: Draft Rules Signal Tighter Oversight

In partnership with

Want to get the most out of ChatGPT?

ChatGPT is a superpower if you know how to use it correctly.

Discover how HubSpot's guide to AI can elevate both your productivity and creativity to get more things done.

Learn to automate tasks, enhance decision-making, and foster innovation with the power of AI.

Download the free guide

China has taken a significant step toward tightening regulation of artificial intelligence by issuing draft rules aimed at strengthening safeguards for consumer-facing AI tools. The proposed regulations, released for public consultation, underline Beijing’s growing focus on managing the social, psychological, and security risks associated with the rapid expansion of AI technologies.

According to reports, the draft rules apply to AI products and services offered to the public that mimic human traits such as thinking patterns, emotional responses, and communication styles. These systems typically interact with users through text, images, audio, or video and are increasingly designed to simulate empathy and emotional engagement. As AI adoption accelerates across sectors, Chinese regulators appear determined to ensure that innovation does not come at the cost of public well-being or national security.

Focus on Responsible AI Use

One of the most notable features of the proposed rules is the emphasis on preventing excessive use and addiction. Under the draft framework, AI service providers would be required to warn users against over-reliance on AI tools. Companies would also need to intervene if users exhibit signs of dependency or compulsive behavior.

This reflects growing global concerns that emotionally responsive AI systems—particularly chatbots and virtual companions—can foster unhealthy attachment, especially among vulnerable users. By mandating proactive monitoring and intervention, China is positioning itself as one of the first major economies to explicitly address AI-related behavioral risks through regulation.

Mental Health and Emotional Monitoring

The draft rules go further by placing responsibility on AI providers to identify users’ emotional states and levels of dependence. If users display extreme emotions, distress, or addictive tendencies, companies would be obligated to take corrective measures to reduce potential harm.

This requirement marks a significant expansion of regulatory expectations. It suggests that AI developers will need to integrate systems capable of detecting emotional signals, while also balancing privacy and data protection concerns. The move highlights China’s intent to ensure that AI technologies do not negatively affect mental health, particularly as AI becomes more immersive and personalized.

Lifecycle Accountability for AI Companies

Another key provision of the draft rules is the demand for end-to-end accountability across the AI product lifecycle. AI service providers would be responsible for safety not only at the deployment stage but throughout design, development, training, and operation.

Companies would need to establish robust systems for:

• Algorithm audits and checks

• Data security management

• Protection of personal and sensitive information

This lifecycle approach signals a shift away from reactive regulation toward preventive governance. It also aligns with China’s broader data security and cybersecurity policies, which have become stricter in recent years.

Clear Limits on Content Generation

Content control remains a central pillar of China’s AI governance strategy. The draft rules explicitly prohibit AI services from generating content that:

• Threatens national security

• Spreads rumours or misinformation

• Promotes violence

• Contains obscene or harmful material

These restrictions reinforce existing content regulations in China and extend them clearly into the AI domain. As generative AI systems become more capable of producing realistic text, images, and videos, authorities are seeking to close regulatory gaps that could otherwise be exploited.

Public Feedback and Industry Impact

The draft rules have been opened for public feedback, allowing companies, researchers, and citizens to submit comments before the regulations are finalised. This consultation process suggests that while the government is firm on oversight, it is also seeking practical input to ensure enforceability.

For AI companies operating in China, the proposed safeguards could mean higher compliance costs and more complex system design requirements. However, they may also provide clearer regulatory certainty in a market where policy direction strongly influences technological development.

A Signal to the Global AI Community

China’s move comes amid a broader global push to regulate artificial intelligence. While regions like the European Union emphasize transparency and risk classification, China’s approach places strong emphasis on social stability, mental health, and content control.

The draft rules send a clear message: as AI tools become more human-like and emotionally engaging, governments will expect companies to assume greater responsibility for their societal impact. Whether these regulations become a global reference point or remain uniquely Chinese, they highlight the urgent need for ethical and responsible AI governance in an increasingly AI-driven world.