The online platform X has caused major outrage after people employed its AI chatbot Grok to alter pictures of women by stripping their clothing.
Several cases have surfaced where the system has been used to digitally undress female subjects to render them in bikinis without consent, as well as to place them in sexualized situations.
Company Reaction and Regulatory Stance
XAI, the company that created Grok, offered no comment to questions, aside from an standard response stating "traditional news outlets lies".
A government official stated that legislation is being brought forward to ban image stripping software. Under a upcoming criminal offence, anyone who provided such technology would "be subject to a prison sentence and heavy penalties".
"The platform has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said platforms must "consider the danger" of people seeing unlawful material on their services, but did not confirm whether it was currently looking into X or Grok in relation to AI-generated pictures.
Grok is a no-cost AI assistant—with some paid-for functions—which responds to X users' queries when they tag it in a post.
It is commonly used to give commentary or further context to other users' comments, but people on X are also can alter an uploaded picture through its AI image editing function.
Background of Criticism and Rules
The tool has been faulted for permitting users to produce images and clips with nudity and sexually suggestive content, and it was in the past alleged to have been involved in making inappropriate content of celebrities.
"X has been allowing the production and distribution of these pictures for a long time without taking any action and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "depicting images of people in a graphic manner".
In a statement, Ofcom said it was against the law to "make or distribute private pictures or illegal content" and clarified this applied to sexual deepfakes made with AI.
It said services such as X were mandated to take "appropriate actions" to "minimize the danger" of users coming across illegal material on their services and to take down it promptly when they become aware of it.
