PART 3: PREVALENCE OF DEEPFAKE DISINFORMATION IN 2024/25 ELECTIONS

Ahead of the 2024 electoral cycle, GenAI platforms and their deepfake technology had been on the market for a little over a year. OpenAI’s ChatGPT leads the global text AI market, while DALL-E leads the market for realistic image generation. Google’s Gemini competes in both text and image synthesis, while Midjourney gains traction for highly realistic images used in both art and misinformation.

Video tools like Synthesia and DeepBrain AI allow for the creation of AI avatars and virtual anchors for media outlets. Apps such as Reface and DeepSwap popularise face-swaps and deepfakes among general users.

In audio, voice cloning tools have become mainstream. ElevenLabs dominates with hyperrealistic, multilingual voices, joined by open-source projects like TorToiSe and VALL-E, which enable voice replication from minimal data. While these technologies aid accessibility and creative work, they can also lead to impersonation scams and disinformation.

Governments, policymakers, academics and experts almost unanimously agree and are concerned that a massive wave of false information is flooding the information ecosystem.

A small number of outliers believe this to be alarmist. In what has become a canonical paper, Felix M. Simon, Sacha Altay, and Hugo Mercier (2023) argue that the threat of widespread misinformation was “overblown,” and list their reasons as follows:

• Quantity of misinformation: GenAI only dramatically lowers the cost and effort required to produce manipulative content; the spread of misinformation has always been limited more by demand than by supply. A larger pool of misinformation does not automatically translate into greater impact if audiences are not receptive.

• Quality of misinformation: Realism is not the sole determinant of persuasiveness. Emotional resonance, narrative fit, and audience predispositions often matter more than surface quality. A slick deepfake might still fall flat if it does not align with what an audience is primed to believe.

• Personalisation of misinformation: While GenAI can lower barriers to producing such content, it does not fundamentally introduce new microtargeting capabilities. The mechanics of tailoring messages to digital identities already existed through online advertising and data analytics.

They were right.

It is worth reflecting on their last point about the personalisation of misinformation. There were concerns that GenAI would enable greater microtargeting of disinformation, a perspective that may have been founded on alarmism. GenAI does not introduce new microtargeting methods; it merely lowers the cost and effort of tailoring messages to inferred digital identities, psychographics, and other similar data. Its current abilities are limited in producing synthetic media that can be tailored based on identified characteristics such as demography, income, location, etc(Simchon et al., 2024).

Microtargeting itself is not inherently wrong; political parties have used it as a tactic to reach potential voters with tailored messaging before and after the advent of the internet. It is when digital identity data is obtained unethically or illegally and used to manipulate voters with personalised, fear-based disinformation, as seen in the Cambridge Analytica scandal, that trouble arises.

This, of course, is a GenAI ability that will need to be monitored in the coming years, and appropriate mitigation strategies will need to be devised and implemented.

The International Panel on the Information Environment (IPIE) – an independent and global science organisation providing scientific knowledge about the health of the world’s information environment – found just 215 instances of AI-generated deepfake electoral disinformation across all 50 countries with competitive national elections in 2024 (IPIE, May 2025).

Separately, the Knight Institute reviewed 78 cases of AI use in the WIRED AI Elections Project,which tracked political AI content during the 2024 global elections, and found no deceptive intent in 39 instances (Kapoor & Narayanan, 2024).

This was much lower than expected, as confirmed by Meta’s reports. Meta closely monitored the potential use of deepfakes by covert influence campaigns and found only incremental productivity and content-generation gains, accounting for less than 1% of all fact-checked misinformation on its platforms.

Unfortunately, other social media platforms did not publish reports detailing the full scope of the proliferation of deepfake disinformation. Even Meta’s was a little thin on the details. Data access remains an area ripe for civil society advocacy. Platforms must publicly furnish complete datasets on the nature and prevalence of all disinformation, including deepfakes, to facilitate independent monitoring and informed interventions. This would include granular data on content reach and engagement, amplification networks, audience exposure and reactions, as well as transparency in detection, labelling, and enforcement measures.

Despite what was relatively low deployment of deepfake disinformation across the globe, it is still worth reflecting on the incidents and their nature, as noted by the IPIE:

• 80% of countries with competitive elections saw GenAI used during campaigns; India and the USA reported the highest number of incidents (30 each). GenAI incidents were in double digits across eight countries (16%), ranging from 10 in France to 30 in India and the USA.

• The countries with the fewest instances (some as low as one) were Belgium, El Salvador, Finland, Lithuania, Madagascar, Maldives, Mauritania, Mozambique, North Macedonia, Palau and Panama.

• The remaining 20% of countries with no identified GenAI usage data are primarily those with smaller populations.

• Explanations for the lack of recorded GenAI usage targeting elections in both groups of countries include less journalistic coverage of these countries’ elections and lower internet penetration rates.

• An additional explanation could be restrictions on freedom of expression, which could lead to less user-generated content.

• The original source of GenAI-generated content was unknown in 46% of cases. Of untraceable cases, 79% involved suspected political manipulation.

• Foreign actors were identified in 20% of cases, all linked to malign uses, such as Russian and Chinese coordination.

• Paid commercial actors were involved in 6% of the cases. In 31% of these cases, paid commercial actors worked in tandem with partisan groups, political parties and candidates, and/or foreign actors, whereas 69% involved paid commercial actors acting alone.

• Constructive uses were also noted: 38% of national party and candidate-related GenAI incidents were beneficial, with 16% focused on civic outreach or accessibility.

Comprehensive data on the detection of electoral disinformation deepfakes during the 2025 election cycle is unavailable, as news coverage has notably declined. However, according to the CIGI (2025), the 2025 global elections confirmed key findings from the IPIE report, namely that:

Ultimately, AI is not a stand-alone disruptor but rather a powerful new layer in existing influence operations, with the potential to outpace rules and regulations if not managed appropriately.

Altay and Mercier argued in 2025, as they had in 2023, that concerns about GenAI threats were exaggerated and cautioned that alarmism over GenAI could distract from ongoing issues, such as deepfakes targeting women and excluded groups. They caution:

By overemphasizing the risks of GenAI in the context of elections, we risk overlooking the broader, more insidious ways in which GenAI is misused, such as enabling targeted harassment and amplifying harmful biases. These include the harassment of women and minorities. The creation and distribution of AI-generated fake nudes, mostly targeted at females, is a form of gendered violence that seeks to silence women in public life and can be used to humiliate, discredit, and threaten women, which may have a chilling effect on their participation in politics. Similarly, minorities are targeted by AI-assisted harassment campaigns, including racially biased or xenophobic attacks that are amplified through social media. These targeted campaigns undermine efforts to build inclusive political spaces.

In addition to the reasons identified by the IPIE, such as low internet penetration rates, limited media coverage, and digital authoritarianism, as well as those highlighted by Simon, Altay, and Mercier, we explore other reasons that could have been resilience factors limiting the spread of deepfake disinformation.

These must not be seen as definitive explanations but rather as a summary of the collective measures implemented before elections that may have played a resilience role. The level of concern and corresponding preparation for disinformation threats in this context was unprecedented. It may signal the need for continued proactive planning.

3.3.1 Public Awareness of Deepfake Disinformation

There appears to have been high level of awareness across the globe of the threat posed by AI and disinformation, as revealed by a 29-country Ipsos survey (Ipsos, 2023). The survey data has limitations that extend beyond the typical constraints of opinion polling and it was conducted in mid-2023, so the conditions and perceptions may have changed since then.

In the survey – Global Views on AI and Disinformation – Ipsos sought to measure how people around the world perceived the risks posed by AI-generated misinformation. It surveyed over 21,000 people in 29 countries, with three-quarters believing that AI makes it easier to produce realistic disinformation and just over half believing it would worsen the problem of disinformation. Despite this, most were confident they could spot fake content, revealing a potential gap between perception and reality. Many also felt that political and media dishonesty have increased over the past thirty years, indicating a broader trend of less reliance on established institutions for information and a growing reliance on sources such as social media influencers for “alternative truths,” which can lead to inadvertent consumption of and belief in misinformation.

The chart below contains the surveyresults for Global South countries, where respondents indicated to what extent they agree or disagree with these two statements: statements

• “Artificial intelligence is making it easier to generate very realistic fake news stories and images.”

• “Artificial intelligence will make misinformation and disinformation worse.”

3.3.3 Regulatory Readiness

Ahead of the 2024 elections, many governments and policymakers acknowledged the dangers posed by deepfake disinformation and began developing strategies to address it. Efforts included detecting, labelling and, in some cases, restricting AI-generated content through both voluntary industry agreements and new laws. These varied initiatives signal growing acknowledgement that deepfakes pose serious risks to election integrity and democracy.

For comparative purposes, the following list of legislation and policies across different jurisdictions includes those that did not hold elections in 2024/25. The elements that worked and did not work could be extracted from each model applied or under consideration. This is another area where civil society advocacy will be particularly beneficial, namely informing of what successful legislation and policies exist or are under consideration.

The 2024/25 election cycle offers numerous lessons. The first is simple: prepare, prepare, prepare. It is far better to over than underprepare.

Early warnings can sometimes cause alarm, but new technological threats demand thorough human impact assessments and awareness of both potential and real impacts. In 2024, what could be termed as alarmism led to public awareness, platform intervention, CSO readiness and regulatory action, which, collectively, may have helped reduce the impact of deepfake disinformation.

Another key lesson is that these assessments must happen before new technologies are widely adopted, not after. Reactive responses are always too late. Anticipatory strategies give societies the resilience to absorb shocks.