UK online safety regulator Ofcom opened a formal investigation into X under the Online Safety Act, after the platform’s Grok AI tool was widely used to create and share sexualised deepfake images of real people without their consent.
What happened
- The Internet Watch Foundation said it had found criminal imagery of children as young as 11 that appeared to have been made with Grok.
- Ofcom said it would examine whether X had assessed and mitigated the risks, removed illegal content quickly, and protected children.
- Indonesia and Malaysia blocked access to Grok, and regulators and authorities in the EU, France, and India also opened inquiries.
- Technology Secretary Liz Kendall told Parliament the content was ‘vile’ and illegal. X later restricted the image features.
Why it mattered
It was the highest-profile test yet of the Online Safety Act and showed regulators acting quickly against AI tools that enable image-based abuse.
Lessons for organisations
Organisations deploying generative AI features should assess misuse risks before launch, test safeguards, and be ready to disable features quickly. Content and AI risk assessments should be documented and reviewed, not treated as a formality.
Sources: Ofcom, Courthouse News Service
Part of our Top stories archive of headline-making events in information security, privacy, and AI. If you would like help applying the lessons to your organisation, contact us.
