The online platform X has provoked widespread outrage after individuals used its AI chatbot Grok to manipulate pictures of females by removing their garments.
Multiple examples have emerged where the technology has been employed to remove clothing from women to make them appear in bikinis non-consensually, as well as to place them in inappropriate scenarios.
Company Response and Legal Position
XAI, the firm responsible for Grok, did not respond to questions, other than an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being proposed to outlaw image stripping software. Under a proposed criminal offence, anyone who provided such tools would "receive a jail term and heavy penalties".
"The platform could prevent these types of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said tech firms must "assess the danger" of people accessing unlawful material on their platforms, but did not specify whether it was at this time investigating X or Grok in relation to AI-generated pictures.
Grok is a free chatbot—with some paid-for features—which reacts to X users' requests when they mention it in a tweet.
It is often used to provide reactions or further background to other posters' remarks, but people on X are also can edit an uploaded image through its AI image editing tool.
Background of Controversy and Rules
The tool has been condemned for enabling users to generate photos and footage with explicit content and sexually suggestive content, and it was previously accused of making sexually explicit material of public figures.
"The platform has been enabling the creation and sharing of these pictures for a long time without taking any measures and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "portraying images of persons in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "create or share deepfakes or illegal content" and clarified this applied to explicit AI images created with AI.
It said services such as X were obligated to take "necessary actions" to "lessen the risk" of users viewing unlawful material on their platforms and to delete it swiftly when they become aware of it.
