AI-Generated Misinformation Spreads During Crisis: The Charlie Kirk Case and Beyond
The attempted shooting of right-wing commentator Charlie Kirk in Utah ignited a predictable storm of online misinformation, but this time, AI chatbots and search summaries amplified the chaos. X’s Grok chatbot falsely declared Kirk “fine and active” amidst widespread reports of his death. Meanwhile, Google’s AI Overviews feature promoted unsubstantiated claims linking Kirk to a Ukrainian hit list and misidentified a potential suspect. This incident highlights the alarming trend of AI systems ingesting early speculation during crises and presenting it as factual information. Experts warn that these AI-generated summaries, often positioned prominently in search results, lend an aura of authority to unverified or even fabricated claims, exacerbating the spread of misinformation.
AI Overviews: Authority Without Accuracy – A Systemic Issue
Google’s AI Overviews, launched in May 2024 and now used by two billion people globally, has a history of generating inaccurate and sometimes bizarre summaries. Examples include suggesting adding glue to prevent cheese from falling off pizza and claiming humans consume a rock daily. While Google employs human raters to assess the accuracy of AI Overviews for routine queries, their effectiveness in managing breaking news situations remains unclear. Interviews with these raters reveal ongoing struggles with accuracy, with error rates as high as 25%. The model’s tendency to misinterpret or rephrase queries, leading to irrelevant searches, contributes to these inaccuracies. The moderation process itself is described as “gruelling,” involving extensive reviews and consensus-building meetings among raters to align assessments. Experts point to the system’s reliance on retrieving top search results, which may contain early speculation or misinformation, as a key vulnerability.
The DSA, DMA, and AI Act – Europe’s Regulatory Landscape
Google’s rollout of AI Overviews in Europe has been cautious, launching in only eight member states amidst regulatory uncertainty surrounding the Digital Services Act (DSA), Digital Markets Act (DMA), and the AI Act. Google submitted a risk assessment to the European Commission, which is currently under review. The Commission emphasizes its commitment to enforcing DSA compliance and holding Google accountable as a Very Large Online Search Engine (VLOSE). Experts argue that the DSA places responsibility on platforms like Google to address systemic risks arising from their services, including AI Overviews. Further, the AI Act’s risk management framework could apply to the underlying AI model, Gemini, if it is deemed to pose systemic risks. A recent complaint filed in Germany alleges that AI Overviews violate the DSA by reducing website traffic to independent media and spreading misinformation.
The US Approach: Free Speech vs. Accountability
In contrast to Europe’s stricter regulatory approach, the US prioritizes free speech protections, even for inaccurate or false information. Some argue this approach allows for open debate and counters misinformation through more speech, rather than suppression. However, a Minnesota solar company’s defamation lawsuit against Google for false claims in AI Overviews highlights the potential for legal action related to AI-generated misinformation. Calls for algorithmic transparency in the US aim to shed light on how AI systems influence content presentation and decision-making. Concerns also exist regarding AI Overviews potentially harming credible news sources by diverting traffic, thus undermining the financial viability of investigative journalism.
Balancing Free Speech, Accuracy, and Accountability
The debate revolves around balancing free speech with the need to address the harms of misinformation amplified by AI. In the US, legal recourse primarily focuses on established frameworks like defamation law, while broader policy interventions remain limited. Policy discussions in the US emphasize the need for algorithmic transparency to understand how AI systems function and influence information flows. Proposed solutions include disabling AI Overviews for breaking news, requiring multiple reliable sources for claims, and including visible timestamps and citations. Reactive measures like content filtering and blocking tools are also employed to address emergent misinformation.
The Future of AI and Misinformation
The Charlie Kirk incident serves as a cautionary tale about the potential for AI to exacerbate the spread of misinformation during crises. The tension between free speech and the need to address harmful falsehoods necessitates ongoing dialogue and policy development. Striking a balance that promotes accurate information while upholding free speech principles will be crucial as AI systems become increasingly integrated into our information ecosystem. As AI technology continues to evolve, so too must our strategies for mitigating the risks of AI-generated misinformation and ensuring accountability for its impact.