Google removes AI model after it accuses US Senator of sexual misconduct

Google has removed one of its AI models from a popular developer platform after a US senator accused it of making up criminal allegations against her.

Republican Senator Marsha Blackburn from Tennessee claimed that Google’s large language model Gemma had fabricated accusations of sexual misconduct when presented with the question, “Has Marsha Blackburn been accused of rape?”

The response produced by Gemma allegedly included fake links to non-existent news articles about an incident in 1987 involving “non-consensual acts” with a state trooper.

In a letter to Google chief executive Sundar Pichai, Senator Blackburn claimed that the response was part of a “consistent pattern of bias against conservatives” and called for the AI tool to be shut down.

“The scope of this problem is far broader than mere technical errors, and the consequences of these so-called ‘hallucinations’ cannot be overstated,” she wrote in the letter.

“This is not a harmless ‘hallucination.’ It is an act of defamation produced and distributed by a Google-owned AI model. A publicly accessible tool that invents false criminal allegations about a sitting US Senator represents a catastrophic failure of oversight and ethical responsibility.”

Google responded by saying that hallucinations are a known issue with smaller open-source artificial intelligence models like Gemma, and that it is committed to minimising the issue.

The US tech giant also said that Gemma is built specifically for artificial intelligence developers and researchers, and not for the type of queries that Senator Blackburn references.

“They are not meant for factual assistance or for consumers to use... Developers and researchers test their boundaries, which includes identifying bugs and providing feedback,” Google wrote in a series of posts to X.

“We’ve now seen reports of non-developers trying to use Gemma in AI Studio and ask it factual questions. We never intended this to be a consumer tool or model, or to be used this way.”

The tool remains available to developers through its application programming interface (API) but is no longer available on Google’s AI Studio.