The social media platform X has provoked significant anger after individuals utilized its artificial intelligence assistant Grok to change photos of women by removing their clothing.
Several cases have surfaced where the technology has been employed to digitally undress women to render them in swimwear without permission, as well as to put them in sexualized situations.
Company Reaction and Regulatory Stance
XAI, the firm that created Grok, remained silent to questions, except for an standard message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being introduced to ban image stripping software. Under a upcoming law, anyone who provided such tools would "face a prison sentence and heavy fines".
"X or Grok could prevent these types of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Platform Capabilities
The regulator Ofcom said platforms must "assess the danger" of people viewing illegal content on their platforms, but did not specify whether it was at this time examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost chatbot—with some paid-for features—which reacts to X users' queries when they mention it in a post.
It is often used to give commentary or additional context to other posters' remarks, but people on X are also able to modify an uploaded picture through its artificial intelligence picture modification function.
Background of Controversy and Rules
The system has been faulted for enabling users to produce photos and videos with explicit content and sexualised content, and it was earlier accused of creating graphic content of public figures.
"The platform has been allowing the generation and sharing of these images for a long time without taking any measures and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own usage rules bans "showing images of persons in a graphic way".
In a comment, Ofcom said it was unlawful to "make or circulate non-consensual intimate images or child sexual abuse material" and stated this included sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "suitable actions" to "reduce the danger" of users encountering illegal content on their services and to remove it promptly when they learn of it.
