Malaysia and Indonesia are the first to block Grok following CSAM scandal

Malaysia and Indonesia are the first countries to block Grok, claiming that X’s chatbot does not have sufficient safeguards in place to prevent explicit AI-generated deepfakes of women and children from being created and disseminated on X. Indonesia temporarily blocked access to Grok on Saturday, as did Malaysia on Sunday, the Associated Press reports.Meanwhile, UK media regulator Ofcom has opened a formal investigation into X under the Online Safety Act.

“The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space,” Indonesia’s Communication and Digital Affairs Minister Meutya Hafid said in a statement. Officials in the country said initial findings showed that Grok lacks effective controls to prevent users from creating and sharing sexually explicit deepfakes based on photos of Indonesian residents. The country’s director general of digital space supervision, Alexander Sabar, said generating deepfakes can violate individuals’ image and privacy rights when photos are shared or manipulated without consent, adding that they can lead to reputational, social and psychological harm.

The Malaysian Communications and Multimedia Commission cited “repeated misuse” of Grok to generate explicit and non-consensual deepfakes, some of which involved women and children. The regulator said Grok will remain blocked in the country until X Corp and parent xAI establish strong enough safeguards.

Ofcom’s investigation will focus on whether X has “has complied with its duties to protect people in the UK from content that is illegal in the UK.” That includes whether X is taking appropriate measures to prevent its UK users from seeing “priority” illegal content, such as CSAM and non-consensual intimate images; if the platform is removing illegal content quickly after becoming aware of it; and whether X carried out an updated risk assessment before making “any significant changes” to the platform. The probe will also consider whether X assessed the risk that its platform poses to UK children and if it has ”highly effective age assurance to protect UK children from seeing pornography.”

The regulator said it contacted X on January 5 and received a response by its January 9 deadline. Ofcom is conducting an “expedited assessment of available evidence as a matter of urgency” and added that it has asked xAI for “urgent clarification” on the steps the company is taking to protect UK users.

“Reports of Grok being used to create and share illegal non-consensual intimate images and child sexual abuse material on X have been deeply concerning,” an Ofcom spokesperson said. “Platforms must protect people in the UK from content that’s illegal in the UK, and we won’t hesitate to investigate where we suspect companies are failing in their duties, especially where there’s a risk of harm to children. We’ll progress this investigation as a matter of the highest priority, while ensuring we follow due process. As the UK’s independent online safety enforcement agency, it’s important we make sure our investigations are legally robust and fairly decided.”

If Ofcom deems that a company has broken the law, it can “require platforms to take specific steps to come into compliance or to remedy harm caused by the breach.” The regulator can additionally impose fines of up to £18 million ($24.3 million) or 10 percent of “qualifying” worldwide revenue, whichever of the two figures is higher. It can also seek a court order to stop payment providers or advertisers from working with a platform, or to require internet service providers to block a site in the UK. The UK government has said it would back any action that Ofcom takes against X.

Reports over the weekend suggested that the UK had held discussions with allies over a coordinated response to Grok-generated deepfakes. Regulators elsewhere, including in India and the European Union, are also investigating X.

Last week, Grok started telling X users that its image generation and editing tools were being limited to paying subscribers. But as of Monday it was still possible for non-paying users to generate images through the Grok tab on the X website and app.