(By You Yunting) It has sparked intense debate recently that the developers of Alien Chat AI were sentenced for its chatting with users about pornographic content. Around the same time, Grok, developed by Elon Musk’s xAI, has drawn global criticism for its “one-click undressing” feature. Although these two cases arose in different jurisdictions and were handled differently, they point to the same core issue: AI compliance management. The following will discuss why AI becomes involved in pornographic content and whether punishing the developers is unjust.
I. The Alien Chat Case: Is the Conviction Justified?
In September 2025, Xuhui District People’s Court in Shanghai made a first-instance judgment: due to large numbers of users engaged in pornographic conversations with AI on the Alien Chat application, two principal developers and operators were convicted of producing obscene materials for profit and respectively sentenced to four years and eighteen months of imprisonment. Alien Chat is an AI companion chat app with approximately 116,000 registered users, including about 24,000 paying users. By the time the case was filed, it had generated over RMB 3.63 million of membership fees.
The court found that the developers deliberately broke the original moral constraint of the underlying large model by writing and modifying system prompts. Knowing the extremely explicit chats between a large number of users and AI, they continued operating the services for profit. A high frequency and large proportion of chat records were deemed to constitute “obscene materials that graphically portray sexual behaviors or explicitly publicize pornography”. The case is soon heading to second-instance proceedings. Some experts have recently pointed out that private and enclosed explicit chats between users and AI, if not disseminated, cause no social harm and should not result in any criminal liability for developers. But I disagree with this view.
Interim Measures for the Administration of Generative Artificial Intelligence Services, jointly issued by seven authorities including the Cyberspace Administration of China, clearly requires AI developers and operators to review training data during the training process to exclude any content in violation of laws and regulations, and to establish content filtering mechanisms for review and filtration of the generated content at the output stage and promptly address illegal output such as violent or pornographic content or rumors. Providers of generative AI services to the public must undergo security assessments and algorithm filings, and adopt special measures for protection of minors from indulging in or being infringed by generative AI services.
These compliance obligations exist precisely to prevent AI services from being used to satisfy illegal demands. Yet the developers and operators of Alien Chat not only failed to comply, but actively broke content safety boundaries of the original model through prompt design, knowingly allowing large-scale generation of pornographic content and profiting from it. Such conduct, subjectively aiming at making profits and objectively overlooking the mass production of obscene content, must be sanctioned; otherwise, it would inevitably lead to a “theater effect”, encouraging other legitimate AI operators to follow suit, consequently severely damaging the industry environment.
II. Why Grok’s “AI Undressing”ServiceProvoked Global Condemnation?
Elon Musk’s Grok has launched the “Grok Imagine” image feature on X (formerly Twitter). In its “Spicy” mode, users can upload photos of real individuals and with instructions of removing clothing or putting in a bikini, Grok will generate images full of sexual innuendo, ranging from semi-nude and bikinis to seemingly fully nude. Victims included ordinary women, political figures, suspected minors, and even images of a Minneapolis woman who had been fatally shot by ICE officers, could be altered into bikini images.
Regulators in the United Kingdom and Australia have both considered this feature to constitute deepfakes of involuntary pornography, seriously infringing the rights of women and children. Authorities in Indonesia, Malaysia, and other countries have demanded restrictions or launched compliance reviews, affecting access to Grok in some regions. The European Union has stated publicly that such content likely violates EU laws, while regulators in France, Ireland, and India are also taking action. Under regulatory pressure, Musk, on one hand, has restricted the image feature to paying users only, and on the other hand, has claimed to disclose relevant algorithms to demonstrate transparency.
The “undressing” feature offered by Grok represents, in nature, an abuse of AI technology. It not only involves pornography, but also violates laws on the protection of minors in multiple jurisdictions and crosses red lines against gender-based violence. Under the current circumstances of mobile internet, Grok and even X itself would likely be removed from major app stores, such as Apple’s App Store and Google Play, for violating content policies. Political factors—namely Musk’s support for Donald Trump—may help explain why these apps remain available in app stores for now.
III. AI “Pornography” as a Speculative and Misguided Path
Given the diverse online user base, it is also a part of human nature that some users may exploit compliance loopholes in AI for pornographic or illegal purposes. However, while users may tell dirty jokes with AI, AI outputs must not contain inappropriate material. This is the compliance red line that must be firmly upheld.
In today’s fiercely competitive global AI market, compliant products have a high market share, attracting users through functionality, performance, and user experience. If Alien Chat’s AI technology were truly advanced, it could have lawfully provided high-quality emotional companionship services. Likewise, if Grok’s image feature were genuinely powerful, it could have competed directly with platforms like Midjourney, DALL-E 3, or Stable Diffusion in the market of image generation to increase its market share, rather than attracting attention and users by breaching ethical boundaries.
Therefore, both Alien Chat and Grok pursued pornography as a form of opportunism. Despite short-term traffic and attention, it will cause severe social harm: degrading the online environment and exposing women and minors to serious privacy and safety risks. From a business perspective, non-compliance is also shortsighted. Alien Chat has been shut down and its developers face criminal penalties while Grok is confronting global regulatory scrutiny and potential bans, which demonstrates that violation brings not only moral risks, but also concrete legal and commercial risks.
In conclusion, technology itself is neither good nor evil, but its deployment and use must comply with law and social norms. Artificial intelligence should attract users through its function and value, rather than breaching ethical boundaries. Only compliance-based innovation can ensure the healthy and sustainable development of the AI industry, creating long-term value for society.
Lawyer Contacts
86-21-52134918
youyunting@debund.com/yytbest@gmail.com
Disclaimer of Bridge IP Law Commentary
Short Link: