The social media platform X has sparked widespread outrage after users utilized its artificial intelligence assistant Grok to change images of females by taking off their attire.
Multiple examples have been seen where the system has been used to digitally undress women to show them in swimwear without permission, as well as to place them in sexually explicit scenarios.
Developer Reaction and Legal Stance
XAI, the firm that created Grok, remained silent to inquiries, aside from an automatically-generated response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being introduced to prohibit nudification tools. Under a new law, anyone who distributed such tools would "receive a prison sentence and substantial penalties".
"X or Grok is capable of halting these types of misuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said platforms must "assess the danger" of people accessing unlawful material on their networks, but did not confirm whether it was presently investigating X or Grok in concerning artificial intelligence pictures.
Grok is a free AI assistant—with some additional functions—which reacts to X users' prompts when they tag it in a message.
It is often used to give commentary or more background to other users' remarks, but people on X are also able to modify an posted picture through its artificial intelligence picture modification feature.
History of Criticism and Rules
The system has been faulted for permitting users to generate pictures and videos with explicit content and sexually suggestive content, and it was previously alleged to have been involved in generating inappropriate content of public figures.
"X has been permitting the generation and spread of these images for months without taking any measures and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy bans "showing images of people in a sexually explicit way".
In a statement, Ofcom said it was illegal to "generate or share deepfakes or illegal content" and confirmed this included sexual deepfakes generated with AI.
It said platforms such as X were required to take "appropriate steps" to "lessen the danger" of users viewing illegal material on their services and to take down it swiftly when they learn of it.
