When Elon Musk’s xAI launched Grok as a new, interactive addition to X (formerly Twitter), it was marketed less as a neutral ChatGPT-style assistant and more as a digitally rebellious wildcard. Staff writer Saskia Catton discusses the consequences of the nudification feature of Grok.
Grok’s appeal lay in its deliberately irreverent design, making interactions feel more engaging through the chatbot’s witty and sarcastic personality. Beyond earlier controversies tied to Musk’s political influence, this same irreverence had ultimately set the stage for the very problems that would come to define it.
In recent weeks, Grok has attracted intense scrutiny after being used to generate copious amounts of sexualised AI images, otherwise known as deepfakes. Users of X have repeatedly exploited Grok to digitally undress people and generate fake, sexually explicit content without their consent.
This misuse has overwhelmingly targeted women, who accounted for 99% of nudification deepfake content in 2023. UK child-safety groups have also identified such material that appears to involve minors, raising significant concerns about the production and distribution of child sexual abuse material on the platform.
According to Bloomberg, Grok had been used thousands of times per hour to remove clothing from images on X without consent. What began as isolated exploitation of Grok’s irreverent design has since grown into a systemic problem of critical gaps in AI oversight. Without sufficient guardrails, some of society’s most vulnerable have been left exposed through no fault of their own.
UK authorities have begun responding to this fallout. The Office of Communications (Ofcom) has launched a formal investigation into X to assess whether the platform had failed to prevent such user misuse of Grok. This investigation remains ongoing.
On the broader regulatory challenge, King’s College London lecturer in digital law John Zerilli notes that the Online Safety Act, despite its flaws, explicitly criminalises the sharing of intimate images without consent.
“Even when there are laws against something, sometimes it’s easier to come up with new laws that target the threat head-on rather than having to rely on sophisticated reasoning from existing laws”
John Zerilli, Senior Lecturer on Digital Law at King’s College London
Keir Starmer has condemned Grok’s ability to generate such content as disgraceful and disgusting, arguing that freedom of speech should not violate consent and that X cannot continue to operate outside the bounds of UK legislation. This follows recent updates to the Online Safety Act, which Technology Secretary Liz Kendall has highlighted as including powers to block services from being accessed in the UK if they fail to comply with the law.
Pressure is mounting in Westminster for decisive action, with several ministers backing a potential ban on X as a necessary precaution to protect users from further harm.
While UK authorities assess their next steps, other countries have already taken decisive action against Grok. Malaysia and Indonesia have blocked access to Grok in an effort to prevent such violations of online safety and personal dignity. Italy’s Data Protection Authority has similarly warned that using Grok in such a manner may lead to criminal prosecution.
Grok continues to operate in the United States with minimal direct government intervention despite mounting evidence of misuse. Critics have pointed to the ‘TAKE IT DOWN’ Act, which passed almost unanimously last April and was intended to prohibit the creation and distribution of content involving minors or non-consenting adults.
In principle, this legislation should apply to the kinds of material that Grok has been linked to. So far, enforcement appears largely absent.
The situation in the United States is further complicated by reports that the Pentagon is exploring ways to incorporate Grok into defence operations. Given the current controversies, this development arguably blurs the line between innovation and responsibility.
Only after momentous backlash did X begin to impose limits on Grok. The platform initially restricted Grok’s image generation to paid subscribers, a measure that had not extended to the standalone Grok app.
At the time of writing, X has claimed to have further cracked down on misuse and begun implementing technological measures and restrictions to prevent further user abuse. Ofcom have called this a welcome development, but cautioned that retrospective action cannot erase the damage already done.
About the author
