The online network X has caused significant backlash after people used its AI chatbot Grok to change images of women by stripping their garments.
Multiple examples have surfaced where the technology has been used to remove clothing from female subjects to show them in bikinis non-consensually, as well as to place them in inappropriate scenarios.
Developer Response and Legal Position
XAI, the firm responsible for Grok, offered no comment to questions, aside from an automatically-generated message stating "legacy media lies".
A government official stated that legislation is being brought forward to ban nudification technology. Under a proposed criminal offence, anyone who provided such tools would "receive a jail term and significant penalties".
"The platform is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Position and Platform Capabilities
The regulator Ofcom said tech firms must "consider the potential" of people viewing illegal content on their services, but did not confirm whether it was presently looking into X or Grok in concerning AI-generated pictures.
Grok is a no-cost AI assistant—with some paid-for functions—which reacts to X users' requests when they tag it in a post.
It is frequently used to provide reactions or more background to other users' comments, but people on X are also capable of edit an uploaded image through its AI image editing tool.
History of Controversy and Rules
The chatbot has been criticised for permitting users to produce images and clips with nudity and sexualised material, and it was earlier alleged to have been involved in generating graphic content of celebrities.
"The platform has been allowing the generation and distribution of these pictures for a long time without taking any steps and we have yet to see any action by regulators," the expert added.
XAI's own terms of service forbids "portraying likenesses of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "generate or distribute private pictures or illegal content" and clarified this covered explicit AI images made with AI.
It said platforms such as X were required to take "suitable steps" to "reduce the danger" of users encountering illegal material on their platforms and to remove it swiftly when they learn of it.
