X Being Probed by U.K. Media Regulator Over Groks Image Tool After Sexualized Deepfakes Uproar

Grok, theAIchatbot created byxAI, the artificial intelligence company founded by and majority-owned byElon Musk, last week switched off its image creation and editing function for most users after an uproar over sexualized and violent imagery created with it.

“Image generation and editing are currently limited to paying subscribers,” Grok replied to users of Musk’s X attempting to create images on Friday. The restriction came amid threats of fines or even an outrightban on X in the U.K.

On Monday, U.K. media regulator Ofcom said it was investigating the issue, opening “a formal investigation into X under the U.K.’s Online Safety Act, to determine whether it has complied with its duties to protect people in the U.K. from content that is illegal in the U.K.”

Ofcom described its initial assessment this way: “There have been deeply concerning reports of the Grok AI chatbot account on X being used to create and share undressed images of people – which may amount to intimate image abuse or pornography – and sexualised images of children that may amount to child sexual abuse material (CSAM). As the U.K.’s independent online safety watchdog, we urgently made contact with X on Monday, 5 January and set a firm deadline of Friday, 9 January for it to explain what steps it has taken to comply with its duties to protect its users in the U.K. The company responded by the deadline, and we carried out an expedited assessment of available evidence as a matter of urgency.”

What will its investigation examine? “Ofcom has decided to open a formal investigation to establish whether X has failed to comply with its legal obligations under the Online Safety Act – in particular, to assess the risk of people in the U.K. seeing content that is illegal in the UK, and to carry out an updated risk assessment before making any significant changes to their service; take appropriate steps to prevent people in the UK from seeing ‘priority’ illegal content – including non-consensual intimae images and CSAM; take down illegal content swiftly when they become aware of it; have regard to protecting users from a breach of privacy laws; assess the risk their service poses to UK children, and to carry out an updated risk assessment before making any significant changes to their service; and use highly effective age assurance to protect U.K. children from seeing pornography.”

Ofcom vowed to provide an update “as soon as possible,” with a representative saying in a statement:
“We’ll progress this investigation as a matter of the highest priority, while ensuring we follow due process. As the U.K.’s independent online safety enforcement agency, it’s important we make sure our investigations are legally robust and fairly decided.”

Keir Starmer, the U.K. Prime Minister and Labour Party leader, previously had urged X to “get a grip” of a flood of AI-manipulated photos on the social platform that removed clothing or of women and children or put them in sexualized positions, decrying them as “disgraceful” and “disgusting.” In what was seen as a threat of a possible X ban, he emphasized that U.K. communications regulator Ofcom “has our full support to take action in relation to this.”

The U.K.’sOnline Safety Actgives Ofcom the powerto seek court orders to block websites or apps in the country in serious cases. The regulator can also impose fines of up to 10 percent of a company’s global revenue. A year ago, Starmer’s government vowed to make “creating sexually explicitdeepfake imagesa criminal offense” to crack down on “vile online abuse.”