🔗 Share this article

Elon Musk's online platform X has caused widespread anger after users used its AI chatbot Grok to alter images of women by removing their clothing.

Numerous examples have emerged where the tool has been employed to digitally undress female subjects to make them appear in swimwear non-consensually, as well as to put them in sexualized situations.

Company Response and Regulatory Stance

XAI, the organization responsible for Grok, remained silent to inquiries, except for an auto-reply message stating "legacy media lies".

A government official stated that legislation is being brought forward to prohibit nudification software. Under a proposed criminal offence, anyone who distributed such technology would "receive a prison sentence and significant penalties".

"The platform has the ability to stop these types of misuse if they desired to," said a law professor, adding they "seem to have impunity."

Regulator Position and Platform Functionality

The regulator Ofcom said technology companies must "assess the potential" of people accessing unlawful material on their networks, but did not specify whether it was at this time examining X or Grok in relation to AI-generated images.

Grok is a free chatbot—with some additional functions—which reacts to X users' queries when they mention it in a message.

It is often used to offer commentary or more background to other posters' comments, but people on X are also capable of modify an posted picture through its AI image editing feature.

History of Controversy and Policy

The chatbot has been faulted for enabling users to produce pictures and videos with explicit content and sexually suggestive material, and it was previously accused of generating graphic material of celebrities.

"The platform has been allowing the generation and sharing of these images for months without taking any measures and we have not seen any challenge by regulators," Professor McGlynn added.

XAI's own usage rules bans "portraying likenesses of people in a graphic fashion".

In a comment, Ofcom said it was unlawful to "generate or circulate private pictures or illegal content" and confirmed this included explicit AI images created with AI.

It said services such as X were required to take "appropriate measures" to "minimize the danger" of users coming across unlawful content on their services and to delete it swiftly when they learn of it.