The Rise of AI-Generated Disinformation in China and the Government’s Response
The proliferation of artificial intelligence (AI) has ushered in a new era of technological advancements, but it has also opened Pandora’s Box of misinformation. In China, this challenge has manifested in a surge of AI-generated rumors circulating online, sparking public concern and prompting authorities to take decisive action. Three prominent cases in recent years highlight the growing threat of AI-fabricated narratives and the government’s efforts to combat them. These cases involved false claims about mortality rates among China’s post-1980s generation, a fictitious ban on food delivery services in Guangzhou, and an exaggerated fine imposed on a fruit vendor in Shandong province. Each of these stories, initially spread through social media and online platforms, captivated public attention before being debunked as AI-generated fabrications.
The first case involved a widely circulated rumor claiming a 5.2% mortality rate among China’s post-1980s generation in 2024. This alarming statistic, falsely attributed to the Seventh National Census, sparked widespread anxiety. However, fact-checking platforms quickly exposed the claim as baseless, revealing that the census data, last updated in 2020, couldn’t be extrapolated to predict figures for 2024. State broadcaster CCTV attributed the erroneous figure to an "AI computation error," highlighting the potential for inaccuracies stemming from insufficient training data or unreliable sources used by AI models. Authorities traced the rumor’s origin to three individuals who were subsequently detained, while six others involved in disseminating the falsehood received warnings. This case underscored the dangers of relying solely on AI-generated information without verification and the potential for such misinformation to amplify falsehoods and fuel public anxiety.
The second fabricated story centered on Guangzhou, the capital of Guangdong province, falsely claiming the city was about to ban food delivery services. This rumor stemmed from a misrepresentation of new regulations introduced in December 2023 regarding electric bicycle use, aimed at enhancing road safety. An organized group, seeking to capitalize on online traffic, used AI to generate articles falsely claiming a complete ban on food delivery. The rumor quickly gained traction, sparking concern among food delivery users and delivery workers alike, who feared potential job losses. This incident highlighted the malicious potential of AI-generated disinformation to disrupt public order and incite unnecessary panic. Authorities identified the perpetrators and initiated criminal investigations, underscoring the government’s resolve to hold those responsible for spreading such rumors accountable.
The third case revolved around a fabricated story about a fruit vendor in Jinan, Shandong province, who allegedly faced a 1.4 million yuan fine for operating without a business license. This story, designed to evoke public outrage, quickly went viral. However, local authorities found no record of such a case, revealing the story as another AI-fabricated narrative. A media company in Changsha, seeking to boost online traffic and revenue, was identified as the source of the fabricated story. Two employees from the company were detained, demonstrating the authorities’ commitment to addressing the misuse of AI for generating false information intended to manipulate public sentiment.
These three cases exemplify the growing concern over AI-generated disinformation in China. The Cyberspace Administration of China (CAC), along with other relevant authorities, has taken a proactive stance in combating this emerging threat. At the 2025 China Internet Civilization Conference, the CAC highlighted these cases as examples of the dangers of AI-generated rumors, emphasizing the need for robust regulatory measures. The government recognizes the potential for such rumors to disrupt public order, incite panic, and erode public trust.
China’s response to the rise of AI-generated disinformation involves a multi-pronged approach encompassing legal and technical safeguards. In November 2022, the CAC issued regulations mandating clear labeling of AI-generated content to distinguish it from real media. This policy places the responsibility on hosting platforms to identify and label such content, remove harmful content, and establish mechanisms for refuting rumors. In March 2025, further guidelines were issued, requiring users and internet service providers to clearly mark AI-generated content across various formats, ensuring traceability and accountability. These regulations aim to create a more transparent online environment and mitigate the spread of AI-fabricated information.
The Chinese government’s proactive approach underscores its commitment to addressing the challenges posed by AI-generated disinformation. By implementing clear regulations and holding perpetrators accountable, authorities aim to curb the spread of false narratives and maintain social stability. As AI technology continues to evolve, the ongoing development and enforcement of these regulations will be crucial in navigating the complex landscape of online information and safeguarding against the potentially disruptive effects of AI-generated rumors. The challenge for China, and indeed the world, is to harness the power of AI for good while simultaneously mitigating its potential for misuse and manipulation. The fight against AI-generated disinformation is a global one, requiring international collaboration and a concerted effort to promote media literacy and critical thinking skills.