Google has pulled public access to its open-weight AI model family, Gemma, from the AI Studio platform after the system reportedly generated false allegations of sexual misconduct against US Senator Marsha Blackburn of Tennessee.
The problem surfaced when the model was asked whether Blackburn had ever faced such an accusation. Gemma responded with a fabricated claim, complete with invented dates and details that did not withstand even basic scrutiny. No such allegation was ever made, and the timeline the model produced was wrong. Gemma also cited news sources that either did not exist or led to unrelated content.
Google said Gemma was never intended as a consumer-facing tool for answering factual questions. It was built for developers, and misuse by general users prompted its removal from the public platform. The model remains available to developers through an API.
In a formal complaint to Google chief executive Sundar Pichai, Blackburn described the incident as defamation, accusing the model of manufacturing serious criminal allegations and linking to fabricated reports. She also pointed to a separate case involving conservative activist Robby Starbuck, who said Gemma had falsely accused him of grave wrongdoing.
A Wider Problem With AI ‘Hallucinations’
The episode underscores a growing concern about AI-generated “hallucinations,” instances in which a system produces convincing but entirely false information. Although Gemma is smaller than the leading large language models, its errors show that the risk is not confined to the biggest systems. Google reaffirmed its commitment to reducing such failures across all its AI products.
The case also raises broader questions about accountability and the boundaries of how these models are used. Gemma was designed as a developer tool, but its accessibility blurred the line around public use, and Google’s swift decision to remove it from public interaction reflects the mounting tension between innovation, safety, and responsible deployment.
As AI tools grow more capable and more widespread, the incident sharpens the case for oversight and safeguards to prevent the spread of harmful or unverified content. Whether it prompts industry-wide standards or formal regulation remains to be seen.
