Grok AI Misuse: India Cracks Down on Obscene Content, Warns X of Legal Consequences

In partnership with

The Future of AI in Marketing. Your Shortcut to Smarter, Faster Marketing.

Unlock a focused set of AI strategies built to streamline your work and maximize impact. This guide delivers the practical tactics and tools marketers need to start seeing results right away:

• 7 high-impact AI strategies to accelerate your marketing performance

• Practical use cases for content creation, lead gen, and personalization

• Expert insights into how top marketers are using AI today

• A framework to evaluate and implement AI tools efficiently

Stay ahead of the curve with these top strategies AI helped develop for marketers, built for real-world results.

Download the Free Report

The rapid rise of artificial intelligence has transformed digital platforms, enabling faster communication, creative expression, and innovative problem-solving. However, alongside its benefits, AI has also opened the door to serious misuse. In a significant move highlighting growing concerns around AI ethics and digital safety, the Indian government has issued a stern notice to Elon Musk-owned social media platform X (formerly Twitter) over the misuse of its AI tool, Grok. The Centre has directed X to remove obscene and sexually explicit content generated using Grok AI within 72 hours, warning that failure to comply could lead to severe legal consequences.

Government Notice and Allegations

The Union Ministry of Electronics and Information Technology (MeitY) issued a formal notice to X after observing widespread misuse of Grok AI. According to the ministry, users have been exploiting the AI tool to create and circulate obscene, nude, indecent, and sexually explicit images and videos, particularly targeting women. These visuals, often generated through fake or anonymous accounts, are reportedly being used to denigrate and humiliate women in a vulgar and derogatory manner.

In its letter to X, the ministry stated that such activities amount to a “gross misuse of artificial intelligence” and reflect serious failures in the platform’s safeguards and enforcement mechanisms. The government emphasised that Grok AI, which is integrated directly into the X platform, lacks adequate checks to prevent malicious and unethical use.

72-Hour Deadline and Compliance Requirements

The government has given X a strict 72-hour deadline to take down all content that violates Indian laws and community standards. Beyond content removal, the ministry has also asked the platform to conduct an immediate and comprehensive review of Grok AI’s technical, procedural, and governance frameworks. This includes evaluating how the AI is trained, how content moderation is handled, and what preventive mechanisms exist to stop abuse before harmful material is published.

Additionally, X has been directed to submit a detailed “Action Taken Report” to the ministry, outlining the steps it has taken to comply with the notice and to prevent future misuse of its AI tools.

Legal Framework and Safe Harbour Warning

The notice cites violations under the Information Technology Act, 2000, as well as the IT (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021. Under these laws, social media platforms operating in India are required to exercise due diligence in preventing the spread of unlawful and harmful content.

Most notably, the Centre warned that non-compliance could result in X losing its “safe harbour” protection under Section 79 of the IT Act. Safe harbour provisions protect intermediaries like social media platforms from legal liability for user-generated content, provided they follow due diligence requirements. If this protection is withdrawn, X could be held directly liable for illegal content hosted on its platform, opening the door to legal action, penalties, and potential bans.

Political Reactions and Calls for Accountability

The issue has drawn strong reactions from political leaders. Union Minister for Electronics and Information Technology Ashwini Vaishnaw stated that social media companies must take responsibility for the content they publish and distribute. Responding to questions on the misuse of Grok AI, Vaishnaw said that intervention is necessary and noted that a parliamentary standing committee has already recommended the introduction of stricter laws to ensure platform accountability.

Rajya Sabha Member of Parliament Priyanka Chaturvedi also raised concerns, writing to the IT minister to seek urgent intervention. She highlighted the alarming rise in cases where AI applications are being misused to create vulgar and morphed images of women, stressing the need for stronger safeguards to protect dignity and privacy in the digital space.

Broader Implications of AI Misuse

This incident underscores a larger global challenge: regulating artificial intelligence in a way that balances innovation with responsibility. While AI tools like Grok are designed to enhance user experience and engagement, their misuse for creating non-consensual explicit content represents a form of digital violence. Such practices can lead to harassment, blackmail, emotional trauma, and long-term reputational damage, especially for women.

Experts argue that AI platforms must adopt stronger content filters, watermarking systems, real-time moderation, and stricter identity verification processes. Transparency in AI training data and accountability mechanisms are also essential to prevent abuse.

What Lies Ahead

The Centre’s action against X signals India’s intent to take a firm stance on AI-related harms and platform responsibility. The case could become a landmark moment in shaping future regulations for artificial intelligence and social media governance in the country. If X complies, it may set new standards for AI moderation. If it does not, the withdrawal of safe harbour protection could significantly alter how global tech companies operate in India.