OpenAI Exposes Chinese Government-Linked Misinformation Campaigns Utilizing ChatGPT
San Francisco, CA – June 6, 2025 – OpenAI, the leading artificial intelligence research company, has sounded the alarm on the escalating misuse of its cutting-edge AI tools, particularly ChatGPT, by groups suspected of being linked to the Chinese government. These groups are allegedly leveraging the powerful language model to disseminate misinformation and conduct covert influence operations across the globe. A comprehensive report released Thursday by OpenAI’s intelligence team details numerous instances of such activities disrupted over the past three months, resulting in the banning of multiple accounts involved in these manipulative efforts. The report highlights the growing sophistication and diversity of these tactics, painting a concerning picture of the evolving landscape of online disinformation.
The investigation, spearheaded by Ben Nimmo, lead investigator on OpenAI’s intelligence team, uncovered four distinct campaigns likely orchestrated by Chinese-backed groups. While each campaign pursued different objectives, they all shared a common thread: the exploitation of ChatGPT in ways that violated OpenAI’s usage policies. These campaigns demonstrate a concerted effort to manipulate public opinion and interfere in democratic processes, raising serious concerns about the potential for AI-powered disinformation to undermine trust in online information.
One particularly noteworthy campaign, dubbed "Sneer Review," involved the use of ChatGPT to generate short comments and fabricate artificial discussions on popular online platforms like TikTok, X (formerly Twitter), Reddit, and Facebook. The topics addressed in these manufactured conversations ranged from criticizing a Taiwan-based video game to generating mixed reactions regarding the closure of the US Agency for International Development (USAID). The hallmark of this campaign was its sophisticated approach to mimicking authentic online engagement. By generating both initial posts and subsequent follow-up comments, the perpetrators created a deceptive illusion of organic conversation, making it more difficult for users to discern the manipulated nature of the content.
Beyond orchestrating public discourse, ChatGPT was also found to have been utilized for internal documentation and evaluation of the influence operations themselves. OpenAI discovered instances where the AI model had been used to generate performance reviews of the campaign, essentially providing internal assessments of its effectiveness. The congruity between online behaviors observed by OpenAI and the processes detailed in these AI-generated reports served as strong evidence of the AI’s involvement in the campaign.
The scope of these operations extended beyond online discussions and performance evaluations. OpenAI’s investigation revealed the use of ChatGPT to generate comments on sensitive US political matters, including criticisms of former President Donald Trump’s trade tariffs. Furthermore, the AI model was employed to facilitate cyber activities, such as modifying malicious scripts, configuring systems for attacks, and building tools for brute-forcing passwords. The adaptability of ChatGPT to various malicious purposes highlights the growing challenge of safeguarding AI technologies from misuse.
OpenAI also uncovered attempts to leverage ChatGPT for intelligence gathering operations. In these instances, fake accounts posing as journalists or analysts engaged with real users online to glean sensitive information. One particularly concerning incident involved the use of AI-generated content in communication related to a US Senator’s correspondence, although OpenAI could not definitively confirm if the fabricated content was actually sent. This incident underscores the potential for AI-powered tools to be weaponized for espionage and political manipulation.
The rise of generative AI, particularly since the launch of ChatGPT in late 2022, has been accompanied by growing concerns about its potential to amplify misinformation and facilitate malicious activities. OpenAI, currently one of the most valuable private tech companies globally, with a recent valuation of $300 billion, is at the forefront of grappling with these complex challenges. The company’s proactive investigation and public disclosure of these findings represent a crucial step towards understanding and mitigating the risks associated with AI-powered disinformation. The findings emphasize the urgent need for continued research, robust safety protocols, and international collaboration to combat the misuse of AI technologies for harmful purposes. The fight against AI-driven disinformation is a critical challenge that demands immediate and sustained attention from the tech industry, policymakers, and the global community.