In 2019, Chinese computer scientist Li Bicheng authored a paper outlining a strategic blueprint for using artificial intelligence to automate sophisticated social media manipulation. Co-authored with a member of the Chinese military’s political warfare unit, the proposal suggested creating vast networks of AI-generated personas that could blend seamlessly into online discourse. Unlike traditional “troll farms,” which are often hampered by human error, language barriers, and limited output, Li’s vision involved autonomous systems capable of maintaining long-term, realistic online identities. These accounts would build credibility through mundane posts about hobbies or families before subtly injecting propaganda, effectively nudging public opinion on sensitive geopolitical issues without detection.
For years, China’s digital influence efforts were marked by low-sophistication tactics, such as awkward, non-native phrasing or posts made during odd hours that signaled state-run intervention. However, the emergence of Large Language Models (LLMs) like ChatGPT has transformed the landscape of potential disinformation. These systems possess the ability to mimic human vernacular, respond to cultural nuances, and even generate hyper-realistic images and video content. RAND researchers warn that if state actors leverage this technology, they can produce an endless stream of authentic-sounding content tailored to the polarized nature of American social media, making the detection of foreign influence increasingly difficult.
The threat is no longer theoretical, as global adversaries have already begun experimenting with AI-enhanced propaganda. Recent investigations uncovered a Russian “bot farm” using AI to manage fake accounts with detailed biographies and followers to sow discord, while U.S. and Taiwanese officials have reported instances of Chinese-linked accounts utilizing AI-generated imagery and synthetic hosts to sway elections. From absurd fake news videos featuring AI-generated figures to manipulated photos of world leaders and natural disasters, the sophistication of these influence operations is climbing. U.S. intelligence agencies have officially noted this trend, warning that the People’s Republic of China may attempt to exert influence over the 2024 American electoral process.
Experts at RAND emphasize that the integration of AI into disinformation strategies represents a fundamental shift in the security landscape. Because these tools are becoming ubiquitous and increasingly difficult to distinguish from authentic human interaction, society must prepare to operate in an environment where the online truth is constantly under siege. Even though Beijing denies involvement in such operations, researchers argue that the U.S. must assume AI manipulation is already proliferating. The ability of these systems to exploit human psychology at scale poses a direct, systemic threat to the integrity of democratic societies.
To combat this, the RAND report advocates for a multi-layered defense strategy. This includes calls for social media platforms to implement more rigorous identity verification and for media organizations to adopt digital watermarking to certify the authenticity of their content. While federal regulations regarding user verification are a point of debate, the researchers stress that time is of the essence. Proactive measures, such as public awareness campaigns and improved platform accountability, are essential to preventing future subversion of the democratic process, though these solutions come with significant societal trade-offs.
Ultimately, the most immediate defense against AI-driven disinformation lies in individual skepticism. As AI technology advances, human sensory perception—which we have relied on for millennia to judge reality—can now be easily deceived. Researchers urge social media users to exercise critical pause, particularly when consuming content designed to trigger immediate outrage. By fostering a culture of caution and questioning the source of provocative information, individuals can become more resilient against the “bait” of automated influence campaigns that seek to dismantle social trust and stability from within.