Recent Developments in the Regulation of Deep Fakes: a Focus on California

Key Points 

• California has attempted to address misinformation and disinformation in relation to elections by signing new AI bills into law 

• In the political context, Deep Fakes have the potential to manipulate voters by using manipulated media, including audio, images, and video 

• In the political context, California’s AB 2839 and 2655 could lead the way in AI regulation  

Deep Fakes are inauthentic because they are manipulated media, and they can easily place anyone into audio, a photo, or video. They can make that person appear to do things that they never did and would never do. AI can synthesize images more rapidly and expedite the Deep Fake creation process. In the political context, we have seen several Deep Fakes pop up, whether it is audio robocalls by President Biden, manipulated video regarding President Obama, or AI-generated images of Vice President Harris. This is a critical time: we are approaching the 2024 Presidential Election in November. What has been done in terms of regulation to quell the spread of disinformation? To answer this question, it is necessary to take a closer look at what has taken place in California. 

Back in 2019, California Governor Newsom signed into law the “first of its kind”, AB 730, prohibiting a person, committee, or other entity, within 60 days of an election where a political candidate appears on the ballot, from distributing with actual malice materially deceptive audio or visual media (images or video) of the candidate with the intent to injure the candidate’s reputation or to deceive a voter into voting for or against the candidate, unless the media includes a disclosure stating that the media has been manipulated. To combat the growing distribution of political Deep Fakes, this law took effect on January 1, 2020. 

How did someone know if there was an instance of materially deceptive audio or visual media? The media would have to be shown to falsely appear to a reasonable person to be authentic, and cause a reasonable person to have a fundamentally different understanding or impression of the expressive content than that person would have if they were hearing or seeing the unaltered, original version of the content. 

However, there is the one exception: any media accompanied by a disclosure stating that the media has been manipulated. Likewise, media distributed for news-reporting purposes is similarly exempt, provided that those distributing the context acknowledge that its authenticity is in question or that it does not accurately represent the speech or conduct of the candidate. And media that constitutes satire or parody is also exempt. 

Candidates who believed that they have been affected would have to show that there has been a violation and can seek injunctive relief or seek general or special damages against the distributing party.  

There was some controversy in response to the bill—to the point where the Institute for Free Speech called AB 730 “a bad omen”.The main concern was that the law might have a chilling effect on political commentary and parody (even though there was an exception built in the bill for parody).  

Recent Enactment  

In September, 2024, Governor Newsom stated the following: 

“Safeguarding the integrity of elections is essential to democracy, and it’s critical that we ensure AI is not deployed to undermine the public’s trust through disinformation – especially in today’s fraught political climate. These measures will help to combat the harmful use of deepfakes in political ads and other content, one of several areas in which the state is being proactive to foster transparent and trustworthy AI.” 

That month, AB 2839 was published in the Legislative Counsel’s Digest. It would be in effect until 2027, and would similarly ban the use of materially deceptive content, which includes audio or visual media (images and video)—regularly referred to as AI-generated Deep Fakes, that is intentionally digitally created or modified, which includes, but is not limited to, Deep Fakes, such that the content would falsely appear to a reasonable person to be an authentic record of the content depicted in the media. 

The bill would also prohibit a person, committee, or other entity from knowingly distributing an advertisement or other election communication that contains certain materially deceptive content with malice, subject to specified exemptions. The bill would apply this prohibition within 120 days of an election in California and, in specified cases, 60 days after an election. The bill would authorize a recipient of materially deceptive content distributed in violation of this law, candidate or committee participating in the election, or elections official, to file a civil action to stop the distribution of the media and to seek damages against the person, committee, or other entity that distributed it, except as specified. The bill would require a court to place such proceedings on the calendar in the order of their date of filing and give the proceedings precedence. 

Governor Newsom recently signed the bill into law. The law took effect immediately as an urgency statute to safeguard voter trust before we approach the 2024 Presidential Election.  

Recent Legal Challenge to AB 2839 

On October 8, 2024, it has been reported that there has been a legal challenge to AB 2839. That’s right, about two weeks after Newsom signed the bill into law, a challenge has been made by a California resident and screenwriter, whose viral Deep Fake video of Vice President Kamala Harris was widely disseminated, and a satirical website. The main concern involves First Amendment free speech rights.      

California’s Attorney General defended the law and argued that AI-generated Deep Fakes constituted a global threat to elections because disinformation campaigns erode public trust. Of the numerous AI bills that Newsom signed into law, AB 2839 will be instrumental as we approach the election. 

However, the law will have no effect since a federal judge halted enforcement of the law. The reasons provided by the judge were that the law was too broad and restrictive. The ruling was considered to be a “blow” to California’s intentions to rein in misleading content on social media ahead of Election Day. Apparently, Elon Musk was happy with the ruling, and shared his thoughts about it on X. It is not clear what will happen next. 

Companion Bill: AB 2655 

Also in September 2024, the California General Assembly passed AB 2655, Defending Democracy from Deepfake Deception Act of 2024. Essentially, the companion bill requires large online platforms to block the posting of election disinformation during specified periods before and after an election, and to label certain additional content as inauthentic, fake, or false. The bill also requires platforms to develop reporting procedures for such content. 

However, the bill would exempt from its provisions a regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, if the publication complies with specified disclosure requirements. The bill would also exempt content that constitutes satire or parody. 

Despite the fact that there is an exemption for satire or parody, Elon Musk was compelled to respond to Newsom’s signing the bill into law on X: “Parody is legal in America”. After this, Newsom and Musk had a back and forth online that simply aggravated their relationship. 

This bill has been subject to a legal challenge as well.  

In both AB 2839 and AB 2655, the legislature declared the following: 

• California is entering its first-ever generative AI election, in which disinformation powered by generative AI will pollute our information ecosystems like never before. Voters will not know what images, audio, or video they can trust 

• In a few clicks, using current technology, bad actors now have the power to create a false image of a candidate accepting a bribe or a fake video of an elections official “caught on tape” saying that voting machines are not secure, or to generate the Governor’s voice telling millions of Californians their voting site has changed 

• In the lead-up to the 2024 presidential elections, candidates and parties are already creating and distributing Deep Fake images and audio and video content. These fake images or files can spread to millions of Californians in seconds and skew election results or undermine trust in the ballot counting process 

• The labeling information required by this bill is narrowly tailored to provide consumers with factual information about the inauthenticity of particular images, audio, video, or text content in order to prevent consumer deception 

• In order to ensure California elections are free and fair, California must, for a limited time before and after elections, prevent the use of Deep Fakes and disinformation meant to prevent voters from voting and to deceive voters based on fraudulent content. Accordingly, the provisions of this chapter are narrowly tailored to support California's compelling interest in protecting its free and fair elections. 

  
  

AB 2355 

Newsom also signed into law AB 2355, which requires disclosure on any campaign advertisements made in whole or in part using AI. More specifically, the bill would require a person, committee, or other entity that creates, originally publishes, or originally distributes a qualified political advertisement to include in the advertisement a specified disclosure that the advertisement was generated, in whole or in part, using AI.  

Any registered voter can bring an action in court seeking a temporary or permanent restraining order or injunction against the publication, printing, circulation, posting, or distribution of any qualified political advertisement that violates these disclosure requirements. 

Results of the legal challenge regarding AB 2839 and 2655 

US District Judge John A. Mendez has held that AI and Deep Fakes pose significant risks, but the law likely violates the First Amendment: 

“Most of AB 2839 acts as a hammer instead of a scalpel, serving as a blunt tool that hinders humorous expression and unconstitutionally stifles the free and unfettered exchange of ideas which is so vital to American democratic debate.”

To that, Izzy Gardon, a spokesperson for Newsom, highlighted that the laws protect democracy and preserve free speech: 

“We’re confident the courts will uphold the state’s ability to regulate these types of dangerous and misleading deepfakes,” he said in a statement. “Satire remains alive and well in California — even for those who miss the punchline.” 

Zooming out 

When examining the larger picture, we see that there was a recent attempt at the federal level in September 2023 with H.R.5586 - DEEPFAKES Accountability Act in the 118th Congress (2023-2024). It will be considered by committee next before it is possibly sent on to the House or Senate as a whole. Only time will tell if this bill makes progress in the future so that the entire country is protected.