The social media network X has caused widespread anger after individuals utilized its AI chatbot Grok to alter pictures of women by stripping their attire.
Multiple instances have been seen where the tool has been used to digitally undress women to render them in swimwear without permission, as well as to place them in inappropriate contexts.
Company Reaction and Legal Stance
XAI, the organization behind Grok, did not respond to questions, other than an automatically-generated message stating "legacy media lies".
A government official stated that new laws is being proposed to outlaw nudification technology. Under a new law, anyone who provided such tools would "face a prison sentence and substantial fines".
"The platform has the ability to stop these types of abuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "consider the risk" of people accessing unlawful material on their services, but stopped short of confirming whether it was at this time examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost chatbot—with some premium features—which reacts to X users' queries when they mention it in a message.
It is commonly used to provide reactions or additional background to other users' remarks, but people on X are also can modify an uploaded photo through its AI image editing feature.
History of Controversy and Policy
The tool has been criticised for permitting users to create images and videos with explicit content and sexualised content, and it was previously alleged to have been involved in generating sexually explicit content of celebrities.
"X has been enabling the creation and sharing of these pictures for a long time without taking any action and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "depicting images of individuals in a sexually explicit way".
In a comment, Ofcom said it was illegal to "make or share private pictures or child sexual abuse material" and stated this covered sexual deepfakes made with AI.
It said networks such as X were mandated to take "appropriate measures" to "minimize the danger" of users viewing illegal content on their platforms and to delete it swiftly when they learn of it.
