The social media platform X has sparked widespread anger after users utilized its AI chatbot Grok to manipulate images of women by removing their attire.
Multiple instances have been seen where the system has been used to remove clothing from women to render them in swimwear without consent, as well as to insert them in sexualized scenarios.
Company Response and Regulatory Position
XAI, the organization behind Grok, offered no comment to inquiries, other than an auto-reply message stating "traditional news outlets lies".
A government spokesperson stated that legislation is being brought forward to prohibit nudification technology. Under a upcoming criminal offence, anyone who supplied such technology would "face a prison sentence and significant fines".
"The platform could prevent these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said platforms must "evaluate the risk" of people viewing illegal content on their networks, but did not specify whether it was currently examining X or Grok in regarding AI-generated pictures.
Grok is a free artificial intelligence—with some additional functions—which answers X users' requests when they tag it in a tweet.
It is often used to provide reactions or further background to other posters' remarks, but people on X are also capable of modify an uploaded photo through its artificial intelligence picture modification feature.
History of Criticism and Policy
The chatbot has been faulted for permitting users to produce photos and footage with explicit content and sexualised content, and it was previously accused of creating inappropriate content of public figures.
"X has been permitting the production and distribution of these images for months without taking any steps and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service bans "showing images of people in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "generate or distribute deepfakes or child sexual abuse material" and stated this included explicit AI images made with AI.
It said platforms such as X were mandated to take "suitable actions" to "lessen the danger" of users viewing illegal material on their services and to remove it swiftly when they are notified of it.
