The online network X has sparked major outrage after people employed its AI chatbot Grok to manipulate images of women by removing their garments.
Several instances have emerged where the technology has been employed to digitally undress female subjects to make them appear in swimwear without permission, as well as to insert them in sexually explicit scenarios.
Company Reaction and Regulatory Position
XAI, the firm responsible for Grok, remained silent to inquiries, other than an standard response stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to prohibit nudification tools. Under a new criminal offence, anyone who supplied such technology would "be subject to a prison sentence and significant fines".
"X or Grok is capable of halting these types of misuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "assess the potential" of people viewing unlawful material on their networks, but did not specify whether it was at this time looking into X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some premium functions—which responds to X users' requests when they mention it in a tweet.
It is commonly used to offer reactions or further background to other users' comments, but people on X are also can edit an uploaded photo through its artificial intelligence picture modification feature.
History of Criticism and Rules
The system has been criticised for allowing users to generate photos and footage with explicit content and sexually suggestive content, and it was previously alleged to have been involved in creating sexually explicit material of public figures.
"X has been allowing the creation and distribution of these pictures for months without taking any measures and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of individuals in a graphic fashion".
In a statement, Ofcom said it was illegal to "create or circulate non-consensual intimate images or child sexual abuse material" and stated this covered explicit AI images created with AI.
It said networks such as X were required to take "appropriate actions" to "minimize the risk" of users encountering unlawful content on their services and to delete it promptly when they learn of it.
