Elon Musk's online network X has provoked major outrage after people used its AI chatbot Grok to change photos of women by taking off their attire.
Multiple instances have surfaced where the technology has been employed to virtually strip women to make them appear in bikinis without permission, as well as to insert them in sexually explicit scenarios.
Developer Response and Regulatory Position
XAI, the firm behind Grok, did not respond to inquiries, except for an automatically-generated message stating "legacy media lies".
A government representative stated that new laws is being brought forward to outlaw nudification tools. Under a new law, anyone who distributed such technology would "receive a prison sentence and heavy penalties".
"X or Grok is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the danger" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was at this time looking into X or Grok in concerning AI-generated images.
Grok is a no-cost artificial intelligence—with some paid-for features—which reacts to X users' prompts when they reference it in a tweet.
It is often used to offer reactions or more background to other users' comments, but people on X are also able to edit an uploaded image through its AI image editing feature.
Background of Controversy and Policy
The system has been condemned for enabling users to create pictures and footage with explicit content and sexualised content, and it was in the past accused of making inappropriate content of celebrities.
"The platform has been permitting the production and distribution of these pictures for a long time without taking any measures and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own terms of service bans "portraying likenesses of individuals in a pornographic way".
In a statement, Ofcom said it was against the law to "create or share deepfakes or child sexual abuse material" and confirmed this applied to explicit AI images created with AI.
It said networks such as X were obligated to take "suitable actions" to "minimize the risk" of users encountering unlawful content on their platforms and to delete it quickly when they become aware of it.
