The online network X has caused major anger after individuals employed its AI chatbot Grok to manipulate pictures of females by removing their attire.
Numerous examples have been seen where the technology has been used to virtually strip women to show them in swimwear without permission, as well as to put them in inappropriate scenarios.
Company Reaction and Regulatory Stance
XAI, the firm that created Grok, offered no comment to requests for comment, aside from an automatically-generated message stating "traditional news outlets lies".
A government official stated that new laws is being proposed to prohibit nudification software. Under a upcoming criminal offence, anyone who supplied such tools would "receive a jail term and substantial penalties".
"The platform has the ability to stop these forms of abuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said tech firms must "evaluate the risk" of people accessing illegal content on their services, but stopped short of confirming whether it was presently investigating X or Grok in relation to artificial intelligence pictures.
Grok is a free artificial intelligence—with some premium functions—which responds to X users' queries when they mention it in a post.
It is commonly used to offer commentary or additional background to other users' comments, but people on X are also able to modify an uploaded picture through its artificial intelligence picture modification tool.
History of Criticism and Policy
The system has been criticised for permitting users to generate photos and videos with nudity and sexually suggestive material, and it was earlier accused of generating inappropriate content of public figures.
"X has been enabling the production and spread of these pictures for a long time without taking any steps and we have yet to see any action by oversight bodies," the expert added.
XAI's own terms of service prohibits "portraying images of individuals in a pornographic manner".
In a comment, Ofcom said it was against the law to "create or share non-consensual intimate images or child sexual abuse material" and confirmed this applied to sexual deepfakes made with AI.
It said networks such as X were mandated to take "necessary actions" to "reduce the danger" of users coming across unlawful material on their services and to take down it quickly when they learn of it.
