The online network X has sparked widespread anger after people utilized its AI chatbot Grok to alter pictures of females by stripping their attire.
Numerous instances have emerged where the tool has been used to digitally undress female subjects to show them in swimwear non-consensually, as well as to insert them in inappropriate scenarios.
Developer Reaction and Regulatory Position
XAI, the organization behind Grok, offered no comment to questions, other than an automatically-generated response stating "legacy media lies".
A Home Office official stated that legislation is being introduced to prohibit nudification software. Under a proposed criminal offence, anyone who supplied such tools would "receive a prison sentence and heavy penalties".
"X or Grok could prevent these types of abuse if they desired to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Stance and Platform Capabilities
The regulator Ofcom said technology companies must "consider the risk" of people accessing unlawful material on their networks, but did not confirm whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a free artificial intelligence—with some additional features—which reacts to X users' requests when they mention it in a message.
It is commonly used to offer reactions or additional context to other users' remarks, but people on X are also able to edit an posted image through its AI image editing feature.
Background of Controversy and Policy
The tool has been condemned for permitting users to produce photos and videos with nudity and sexualised material, and it was earlier alleged to have been involved in making sexually explicit material of celebrities.
"The platform has been allowing the production and distribution of these pictures for months without taking any action and we have not seen any challenge by regulators," the expert added.
XAI's own terms of service bans "portraying likenesses of persons in a graphic way".
In a statement, Ofcom said it was unlawful to "make or share non-consensual intimate images or illegal content" and clarified this applied to sexual deepfakes made with AI.
It said networks such as X were obligated to take "appropriate steps" to "reduce the risk" of users viewing unlawful content on their services and to take down it quickly when they learn of it.
