Columbia Journalism Review’s “Ask Anika” column recently took up a question that has become increasingly urgent for reporters and newsrooms: Chatbots can be “poisoned” by fake facts, often spread by influence campaigns or malicious sources, and what can journalists do about it? The columnist, an Ivy League professor and former technology worker who now runs an academic center at Columbia’s Newmark Center for Journalism Ethics and Security, began by recounting a conversation from a few years ago. Amid the initial hype over ChatGPT, a colleague asked for her wildest theories about generative AI vulnerabilities. Without hesitation, she imagined a world in which bad actors could effortlessly publish false information on legitimate-looking websites and thereby change chatbots’ results. She was once told by a tech company CEO that she had “a problem hypothesizing issues,” but the fear was far from paranoid. Last month, The Guardian reported on an influence operation designed to prime chatbots to make pro-Israel arguments, initiated by an Israeli-funded entity that purported to be a United States–based think tank. Over nine days, the fake think tank’s website churned out more than half a million words and a hundred “reports.” Seventy-three of those reports were published within a two-day span, none with bylines. Nick Cleveland-Stout, a research associate in the Democratizing Foreign Policy program at the Quincy Institute, wrote about the operation and found that the site had been created by an advertising firm paid nearly a million dollars through subcontracts by the Israeli government. The episode, the column suggests, was exactly the kind of attack she had imagined: cheap, quickly scalable, and designed not for human readers but for the algorithms that decide what chatbots will repeat.
Such tactics are often described as “generative engine optimization,” a play on search engine optimization, but the column argues that this term belies the real damage. When a chatbot ingests information from the internet, or from prompts typed into its box, the AI company keeps it forever. The information bores deeper, becomes part of the model’s training data, and eventually appears in outputs stripped of citations and sources. This is a journalist’s worst nightmare, because it is impossible to fact-check a statement whose origins have been erased. As Cleveland-Stout told The Guardian, “You won’t be able to fact-check it.” The implications extend far beyond one incorrect chat response. A poisoned chatbot is not simply giving a wrong answer; it is becoming a tool for laundering propaganda into the historical record. Once falsehoods are embedded in a large language model, the model will retrain on those falsehoods, and every future user who relies on it for news and information will receive a distorted version of events. The column warns that if covert influence operations are left to roam freely through the web, the rising number of people using LLMs for news will mostly encounter what it calls “propagandized slop.” The problem is not an abstract or hypothetical concern; it is happening now, and the architectural design of AI systems makes it particularly difficult to undo.
The column’s most controversial recommendation is that journalists and media scholars must work together with technologists and technology companies. This may feel like an uneasy alliance, given the history of friction between the news industry and Big Tech, but the author insists that the professions, the information ecosystem, and society writ large will not thrive if they do not adapt to the current reality. Journalistic reporting has become a main character in a global war on truth, and refusing to engage with AI builders would only leave the field open to bad actors. The column is careful to say that collaboration does not mean abandoning the fight for protections and guidelines in newsrooms, or giving up on compensation and credit for work that is being pilfered at an unprecedented scale. Journalists should continue those fights, but they must also, as the author puts it, “walk and chew bubble gum.” That means maintaining rigorous reporting standards while simultaneously developing the skills and partnerships needed to ensure reliable journalism surfaces in AI outputs. It means treating technology not just as a threat but as a territory to be defended. If newsrooms simply observe AI training from the sidelines, there is no reason to believe chatbot outputs will favor journalism over propaganda. The alternative—letting covert influence operations dominate AI story optimization—would be disastrous for an already fragile information ecosystem.
In practical terms, the column argues that the work of journalists themselves does not change. Reporting, investigating, writing, editing, and publishing facts should remain rigorous and ethical. But news organizations must acknowledge that, just like savvy adversaries, journalists need technologists with novel ideas for AI-powered distribution. If experimentation and mastery of “AI story optimization” are left to influence operations, chatbots will continue to retrain on false information and corrode the facts of history. Newsrooms, therefore, should not merely be consumers of AI tools; they should be active participants in the design of trustworthy information systems. This could take many forms: news organizations might create their own optimized channels for chatbots, develop standards for machine-readable trust signals, or build features that allow LLMs to cite credible reporting rather than anonymous sludge. The column’s message is that journalists cannot simply monitor the crisis from the outside. They have to get their hands dirty, working with engineers, data scientists, and product managers to ensure that real news is what chatbots surface. That may require new roles, new training, and new editorial workflows, but it is an essential investment in the future of journalism. Without journalistic input, there is no reason to expect that AI companies will accidentally develop a respect for verified facts. Editorial judgment has to be built into the systems, not assumed after the fact.
The column also directs specific advice to technology companies. Two years earlier, the author had argued in CJR that builders of large language models should work with journalists to fix how they handle breaking news. Since then, agreements between newsrooms and tech companies have proliferated, and the author predicted that such partnerships would form the basis of a new technological era in which legacy media organizations function as wire services for AI chatbots, providing trusted and timely information that can easily be algorithmically surfaced. But the column warns that there are always human fingers tipping the scales. LLMs determine which information on the internet is reliable enough to display based on instructions, and those instructions contain definitions and metrics for classifying and amplifying what counts as “news,” “trustworthy,” “credible,” or “true.” The author knows from working in tech that many of these instructions are based on policies written by people with no journalism background, training, or even basic knowledge of journalistic essentials. This, she says, is not an ideal situation—but it is easily solved with a little will and collaboration. She points to companies like NewsGuard, which have published rating and scoring criteria for identifying reliable information, and suggests that AI companies could adopt those criteria or ask their chatbots to read and implement them. Academic centers like Columbia’s Newmark Center can and do consult with tech companies to develop better policies for determining whose voices should be amplified, particularly in a world of “news influencers.” Even common sense would help: any website that publishes a torrent of articles with no bylines should be automatically suspect.
Finally, the column acknowledges the uncomfortable reality that technology companies may not have strong incentives to care about the truth. The author says she is not naive enough to believe otherwise, but she offers the column as both a permission slip and a resource: tech workers concerned about the impact of LLMs on the exceedingly fragile information ecosystem can cite it in Slack messages to their trust-and-safety policymakers. She also concedes that collaboration may be uncomfortable for both journalists and technologists. But the need is urgent. Journalists must understand how AI works, and technologists must be able to fathom the work of journalism. Together, they can put on their “white hats” and engineer solutions. The column dismisses apocalyptic visions of AI annihilating humanity in a vague, hypothetical catastrophe. The more risky scenario, as she frames it, is allowing would-be god machines to continue their unfettered engagement in today’s information war. The damage is not coming; it is already underway. The piece was produced with support from the Craig Newmark Center for Journalism Ethics and Security, and it ends as a call to action: if journalists and technologists do not step into this problem together, the information ecosystem will be shaped by the actors who poisoned it. The choice is not between the legacy media and the platforms, but between a future where trustworthy journalism is surfaced and one where propaganda slop wins by default.