The social media platform X has provoked significant backlash after users employed its AI chatbot Grok to change pictures of women by stripping their garments.
Multiple examples have surfaced where the tool has been employed to remove clothing from female subjects to show them in swimwear non-consensually, as well as to insert them in sexualized scenarios.
Developer Reaction and Regulatory Position
XAI, the firm that created Grok, did not respond to inquiries, except for an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being introduced to ban nudification software. Under a new criminal offence, anyone who provided such tools would "receive a prison sentence and heavy penalties".
"X or Grok is capable of halting these types of misuse if they wanted to," said a law professor, adding they "seem to have impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said technology companies must "consider the risk" of people viewing unlawful material on their platforms, but stopped short of confirming whether it was currently investigating X or Grok in regarding artificial intelligence images.
Grok is a free AI assistant—with some additional functions—which answers X users' requests when they reference it in a tweet.
It is often used to offer reactions or further context to other posters' remarks, but people on X are also can modify an uploaded photo through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The tool has been criticised for allowing users to generate images and clips with nudity and sexualised content, and it was in the past alleged to have been involved in making graphic content of public figures.
"X has been permitting the generation and spread of these pictures for months without taking any steps and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "portraying images of persons in a graphic manner".
In a statement, Ofcom said it was against the law to "generate or distribute deepfakes or illegal content" and stated this included explicit AI images generated with AI.
It said networks such as X were obligated to take "suitable steps" to "lessen the danger" of users viewing illegal content on their platforms and to remove it quickly when they learn of it.
