The online platform X has caused widespread backlash after users utilized its artificial intelligence assistant Grok to change images of females by removing their clothing.
Numerous examples have surfaced where the system has been used to digitally undress women to make them appear in bikinis without consent, as well as to insert them in inappropriate situations.
Developer Response and Legal Position
XAI, the company responsible for Grok, offered no comment to requests for comment, other than an auto-reply message stating "legacy media lies".
A government representative stated that new laws is being proposed to prohibit nudification tools. Under a upcoming criminal offence, anyone who provided such tools would "receive a prison sentence and significant fines".
"X or Grok has the ability to stop these types of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Stance and Platform Functionality
The communications watchdog Ofcom said platforms must "evaluate the potential" of people viewing unlawful material on their networks, but did not specify whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost artificial intelligence—with some additional features—which answers X users' requests when they tag it in a message.
It is commonly used to offer reactions or additional background to other users' comments, but people on X are also able to modify an posted picture through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The chatbot has been criticised for enabling users to create photos and footage with explicit content and sexually suggestive content, and it was in the past accused of creating graphic material of celebrities.
"The platform has been enabling the production and sharing of these images for a long time without taking any measures and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "portraying likenesses of people in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "generate or distribute non-consensual intimate images or illegal content" and clarified this applied to sexual deepfakes created with AI.
It said networks such as X were mandated to take "necessary measures" to "lessen the risk" of users coming across illegal material on their platforms and to remove it promptly when they are notified of it.
