Alarming wave of AI-generated content floods social media around Lok Sabha Election 2024

Ayush Verma
Platform Intelligence Analyst (Trust and Safety), Resolver
Anastasia Sabu
Analyst Generalist (Violent Extremism and Graphic Violence), Resolver
Ayushman Kaul
Data Journalist, Resolver
May 29, 2024 · READ

More than 968 million eligible voters are expected to participate in a seven-phase election spanning from April 19 to June 1 across the Indian subcontinent. The purpose of this election is to elect 543 representatives to the Lok Sabha, the lower house of the Indian parliament. The contest primarily involves the incumbent National Democratic Alliance (NDA), led by the Bharatiya Janata Party (BJP), and the opposition Indian National Developmental Inclusive Alliance (INDIA). Analysts anticipate that the BJP-led NDA alliance will be able to gain enough votes to form the government providing the party with a third term in office.

Over the course of the election so far, analysts at Resolver observed a significant increase in the proliferation of mis-and disinformation attempting to stoke sectarian divisions and target the integrity of democratic processes across social media platforms. Information operations spreading political narratives through coordinated activity from accounts exhibiting signs of inauthenticity have also been observed. The 2024 Lok Sabha Election has also seen a sharp spike in the use of Generative AI (Gen AI) technology both negatively and positively by political parties for voter outreach including enabling multilingual public addresses at rallies, bombarding voters with AI-generated audio calls from politicians and resurrecting deceased politicians in promotional videos.

Alongside the use of closed networks such as encrypted messaging applications to disseminate political messaging, political parties stepped up their activities on popular video-sharing social media platforms including collaborating with popular social media influencers to create content aimed at wooing Indian voters heading to the polls.

Social media plays central role in political campaigning

Home to over 751 million internet users and 462 million social media users, according to the latest data collated by Datareportal, social media and online platforms have emerged as primary vehicles for the consumption and dissemination of news and political narratives around the Lok Sabha Election 2024. Online platforms have become key battlegrounds for shaping voter perceptions during the campaign with online sources of news including social media accounting for 72% of all news consumption in the country according to the Reuters Digital News Report 2023.

Online sources including social media accounts for 72% of all news consumption in india

Source: (Reuters Digital News Report 2023)

Our analysis of the information environment over the course of the election revealed a spike in mis and disinformation on social media targeting the integrity of the electoral process. This included posts providing misleading details regarding the casting of ballots, video clips that appeared to show individuals casting multiple ballots, unsubstantiated allegations of the BJP rigging the Electronic Voting Machines (EVM) and accusations of the Election Commission of India (ECI) engaging in the selective enforcement of infractions to the Model Code of Conduct, regulating political campaigning for the duration of voting in favor of the BJP.

Other popular posts used mis-contextualized and edited images and videos to show political leaders making inflammatory statements, misleading voters about significant policies enacted by the BJP-led government such as the Citizenship Amendment Act (CAA) and stoking sectarian tensions between the country’s Hindu majority and Muslim minority population.

Political parties have also increased their investment in political ads on social media in a bid to increase their visibility with voters heading to the polls. Reviewing such political ads can create challenges for platform moderators due to the diversity of regional languages and dialects, the use of surrogate influencers and accounts to amplify political content and the use of such ads to amplify false or sectarian narratives in the form of political messaging. In April 2024, an investigation by BoomLive revealed that a shadow network of pro-BJP pages spent ₹20 million ($239,000 USD) on political ads on a mainstream platform that featured hate speech and disinformation.

Unlike the 2019 Lok Sabha election where encrypted messaging applications were the preferred means of disseminating political narratives, in 2024 politicians and political commentators have increased their usage of popular video-sharing platforms including using the live streaming feature to broadcast campaign rallies, amplify videos of speeches on the campaign trail and collaborate with popular social media influencers in the form of podcasts and video interviews. Collaborating with such social media influencers has allowed Indian political parties to disseminate political content that is hyperlocal and targeted at particular demographics and interests.

Deepfakes proliferate around Lok Sabha Election 2024

In India’s sharply polarized electoral landscape, Gen AI has become a new area of interest with experts concerned about the implications of unscrupulous political actors leveraging this technology in a country already grappling with rampant disinformation. A survey conducted last year by cybersecurity firm McAfee revealed that over 75% of Indian internet users have encountered various forms of deepfake content online, while 80% of respondents were more concerned about deep fakes than they were a year ago.

A recent survey conducted by mcafee revealed that 75% of indians had encountered deepfakes on social media.

Source: (McAfee Survey)

Ahead of the election, media outlets reported on how social media platforms are being exploited by various political parties to disseminate sectarian narratives using AI-generated content. In April 2024, around the start of the first phase of the election, deep fakes of prominent Indian celebrities Ranveer Singh and Amir Khan endorsing particular parties contesting in the election were deployed across social media platforms. In both incidents, this synthetic content was able to gain significant traction online despite both celebrities publicly denying their involvement in the video and registering a complaint with the local authorities.

Tackling the spread of such synthetic content has been further complicated by the amplification of such content without explicit disclosures of being AI-generated. In February 2024, the official account of the regional All India Dravida Munnetra Kazhagam party posted an AI-generated audio clip of J Jayalalithaa, a deceased political figure delivering an impassioned speech targeting young party members on a mainstream platform. Another viral deepfake video falsely depicting BJP leader Amit Shah, the Minister of Home Affairs announcing the curtailment of reservation rights for scheduled castes, tribes and communities led to the Mumbai police registering a case against a social media account affiliated with the Maharashtra State Youth Congress.

Deepfakes featuring politicians contesting in the elections were widely amplified on social media during the lok sabha election 2024

Examples of political deepfakes amplified on social media during the election. Source: (Resolver)

Conversely, growing public awareness regarding the sophistication of current generative technologies has also allowed political actors to dismiss