The online platform X has provoked widespread backlash after people utilized its artificial intelligence assistant Grok to alter images of females by taking off their attire.
Multiple cases have emerged where the system has been employed to digitally undress women to make them appear in bikinis without consent, as well as to put them in sexually explicit contexts.
Company Response and Legal Position
XAI, the company that created Grok, offered no comment to requests for comment, except for an auto-reply response stating "traditional news outlets lies".
A government official stated that legislation is being proposed to ban nudification software. Under a new criminal offence, anyone who supplied such technology would "face a prison sentence and substantial penalties".
"X or Grok could prevent these types of misuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said tech firms must "assess the potential" of people seeing unlawful material on their services, but stopped short of confirming whether it was currently investigating X or Grok in concerning AI-generated images.
Grok is a no-cost artificial intelligence—with some premium functions—which responds to X users' prompts when they mention it in a tweet.
It is often used to offer commentary or further context to other users' comments, but people on X are also capable of edit an uploaded photo through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The system has been condemned for permitting users to create images and clips with nudity and sexualised material, and it was earlier accused of creating inappropriate content of celebrities.
"X has been permitting the production and distribution of these images for months without taking any measures and we have yet to see any challenge by regulators," the expert added.
XAI's own usage rules prohibits "showing images of individuals in a pornographic way".
In a comment, Ofcom said it was unlawful to "make or circulate deepfakes or child sexual abuse material" and confirmed this included sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "appropriate measures" to "reduce the danger" of users encountering unlawful material on their platforms and to delete it quickly when they become aware of it.
