AI-Generated Submissions Flooding Australian Parliamentary Inquiries, Investigation Finds
Australia’s parliamentary inquiry system, designed to strengthen democracy by hearing from experts and constituents before laws are made, is being overwhelmed by artificial intelligence-generated material that invents studies and attributes nonexistent research to real academics, Guardian Australia can reveal. An investigation has found that committee reports have in some cases cited submissions in which the majority of sources appear to be AI-generated “hallucinations” – confident but fabricated output produced by large language models (LLMs) such as ChatGPT. The phenomenon poses a significant risk that parliamentarians will make decisions based on evidence that does not exist, according to Christian Downie, a professor at the Australian National University’s school of regulation and global governance. The problem is compounded by AI-powered search tools, which can make fake references appear legitimate. Where previously a simple online search would quickly reveal an invented citation, Google’s AI summarisation feature now sometimes creates summaries of fake references as though they were genuine academic papers, creating a self-reinforcing cycle of misinformation. The findings raise urgent questions about whether governments have the tools and resources to identify AI-generated errors in material intended to influence policy, and about the wider threat to public trust in democratic institutions.
The investigation uncovered a striking example in a submission to an inquiry into family violence and suicide, which contained a hallucinated reference attributed to Divna Haslam, a University of Queensland associate professor and clinical psychologist, while misstating her team’s research findings. Haslam, who researches child and family adversity and maltreatment, said the reference was “scary” because it looked so real that a person taking a cursory look could be convinced, and that Google’s AI summary had summarised the fake reference as if it were a real paper. “It’s very frustrating to have invested time, money, effort, expertise in rigorous research,” Haslam said. “Then to see something that’s inaccurate and inappropriately attributed anyway is really concerning.” The author of the submission, Drilldown Reports, told Guardian Australia it had used AI in its research process and had already identified the errors in a follow-up submission, but the deadline for corrections had passed. A spokesperson acknowledged the importance of human oversight in quality review but insisted the failure to upload the corrected document was human error, not AI. “I get that you feel you have a juicy article to add the AI fear machine, but this was a human error, not AI,” the spokesperson said. The chair of the parliamentary committee overseeing the inquiry said committees received a range of materials of varying quality and that it was their job to accept evidence on face value but also interrogate it through the inquiry process.
The Guardian’s investigation used a bespoke computer program to extract references from all inquiry submissions made to the current parliament and checked those references against online academic databases, including CrossRef and Google Scholar, as well as verifying digital object identifiers. Documents with a high proportion of references that could not be matched were then manually checked, and a large number of submissions with incorrect references were verified with their original authors. This conservative methodology, which can only point towards AI usage in documents that contain references, still identified at least 39 submissions to political inquiries containing what appear to be hallucinated references. In addition, more than 100 papers included ChatGPT URL tags in their reference links – tags automatically added by the AI platform when it provides links to users, and which persist when copied into documents. The submissions containing fabricated material ranged from documents with a small number of incorrect citations to those in which every reference cited did not exist, and were authored by individuals and organisations across the political spectrum. Many of those responsible were unaware that AI could generate false information in this way, or had known about the potential for errors but missed them before submitting their documents.
Several other examples illustrate the breadth of the problem. A submission to an inquiry into housing inequity this year included references to work by Nicole Gurran, an academic at the University of Sydney, that does not exist. Gurran, a professor of urban and regional planning, said fake citations undermine the transparency and contestability that are vital to research and peer review. “Fake citations, even if an accidental ‘collage’ where the claim is correct but the chain of reference to evidence is broken, undermines that,” she said. In another case, a paper submitted last year to a Senate inquiry into climate change misinformation by the National Rational Energy Network described Guardian Australia as “left-leaning, activist-oriented” and appeared to attribute that claim in part to an “article” by Margaret Simons, a journalist and academic, in a top journal. Simons, who is on the board of Guardian Australia’s owner, the Scott Trust, never wrote such a paper. The endnote was so convincingly formatted that Simons said she briefly wondered whether it really existed. “Even though I know I didn’t write this, I did have that moment of self questioning,” she said. The National Rational Energy Network did not respond to a request for comment.
The implications extend far beyond individual misattributions. Deloitte, the consultancy giant, was forced to issue a partial refund to the federal government last year after it was revealed that AI tools had included fake references and even a fake court reference in a $440,000 report. Downie warned that if public documents such as government submissions or court judgments are found to contain fake material, the consequences could include not only bad policy decisions but a broader erosion of confidence in institutions that underpin democracy. He said the inquiry process needs to remain as open as possible, but that new guidelines may be needed to encourage truthfulness. “Whether it’s climate change, immigration, health, we want our elected officials to be making decisions based on real information and real evidence not on fake citations and fake claims,” he said. Senate advice for inquiry submissions already warns that use of AI can present risks to the quality of information and that accuracy is the responsibility of the submitter, but the investigation suggests this guidance is not being followed in practice. Downie stressed that AI-generated errors are not merely a technical nuisance but a direct threat to the integrity of policymaking processes that rely on genuine evidence.
OpenAI, the maker of ChatGPT, said addressing hallucinations was an ongoing area of research and that users should “use ChatGPT as a first draft, not a final source”, ensuring that they verify quotes, data or references to external documents. A Google spokesperson said AI Overviews operate “like traditional Search” in that they aim to match content from the web to the words in the query: “AI Overviews – like traditional blue link results – will find and surface the web pages that include those terms.” However, the investigation shows that because AI-generated hallucinations are now being cited by other AI tools, fact-checking has become increasingly difficult, with fake references creating a loop of misinformation that can be nearly impossible to break. The chair of the standing committee on social policy and legal affairs said all committees receive a range of materials of varying quality and positions, and that it is the job of committees to interrogate evidence through the inquiry process. Haslam warned that in sensitive areas such as domestic violence, the risk is simply too great. “Particularly in a [domestic violence] space, we just can’t risk that,” she said. As AI becomes more widely used, the challenge for governments will be to preserve the open, consultative nature of parliamentary inquiries while ensuring that the evidence base for legislation remains real, accurate and trustworthy.