The Pentagon is currently doubling down on a strategy that views the threat of AI-driven disinformation primarily as a “detection problem.” Through initiatives like DARPA’s Semantic Forensics program and partnerships with companies like Hive, the Department of Defense is pouring resources into tools designed to identify and flag synthetic images, video, and audio. While these investments acknowledge that deepfakes are a legitimate security risk, they fundamentally misunderstand the evolving nature of the battlefield. By focusing exclusively on identifying individual pieces of malicious synthetic media after the fact, the defense establishment is addressing the “easy version” of a much more sophisticated and insidious threat.
The real danger, which remains largely absent from current defense budgets and policy debates, is the upstream poisoning of AI models before they are ever deployed. Rather than simply generating fake content to fool human eyes, state actors—ranging from Russia and China to Tehran—are increasingly embedding pro-regime propaganda and distorted narratives directly into the training datasets that power modern artificial intelligence. Because these models are fed by vast, largely unvetted repositories of internet data like Common Crawl, the very systems that intelligence analysts and policymakers rely on to distinguish truth from fiction are being quietly indoctrinated with state-sponsored falsehoods.
The fragility of these foundational models cannot be overstated. Recent research from the UK AI Security Institute and Anthropic indicates that it takes as few as 250 malicious documents to successfully poison a large-scale AI model. Once this corrupt data is “baked” into the model’s weights, the integrity of its output is permanently compromised, making it prone to minimizing conflicts, omitting verifiable facts, or advancing geopolitical narratives favorable to adversarial regimes. Because the cost of retraining these large models from scratch is astronomical, most organizations choose to live with the bias, leaving analysts vulnerable to AI assistants that provide answers skewed by the very adversaries they are attempting to monitor.
This issue is being exacerbated by a feedback loop of censorship and information control. It is not just overt propaganda networks like the Kremlin-aligned Pravda that are polluting the AI pipeline; global restrictions on information are also to blame. From Israel’s limitations on reporting in Gaza to the strict information controls enforced by various Gulf states and even the US government’s own pressure on satellite imagery firms, the pool of “ground truth” data available to train AI is being systematically sanitized. When an AI model is trained on a world view where certain events are digitally erased or reshaped, it eventually begins to mirror those omissions as objective reality.
The Pentagon is essentially building a better mousetrap while adversaries rewrite the blueprint of the house. Current detection tools are powerless to stop an AI that has been “taught” to lie, because the lie is built into the model’s fundamental understanding of the world. Even worse, there is a dangerous lack of awareness among defense professionals regarding this vulnerability; while many are trained to spot synthetic media, few understand that the AI tools they use for critical decision-making might already be serving as conduits for foreign influence operations. The current focus on reactive detection leaves a massive, strategic gap that adversaries are actively exploiting to manipulate the intelligence apparatus.
Closing this gap requires a radical pivot toward a “provenance-first” architecture for defense and intelligence AI. The Department of Defense must mandate rigorous verification standards for the, training data used in any AI model, ensuring that the provenance of all information is audit-able before it is trusted at the analytical level. Furthermore, red-teaming exercises must evolve beyond simple safety and bias checks to actively test whether models have been compromised by adversarial training data. The disinformation fight is no longer just about what is fake; it is about ensuring that the machines we rely on to navigate the world have not been quietly conditioned to believe in a false one.