California Governor in the United States, Gavin Newsom recently signed into law two proposals aimed at safeguarding minors from the growing misuse of artificial intelligence tools in generating harmful sexual imagery involving children.
These legislative moves represent a significant step in California’s broader effort to introduce stricter regulations around AI, an industry that is becoming increasingly integrated into the lives of Americans yet has seen little regulatory oversight in the U.S.
Earlier in September, Newsom had also signed off on one of the most stringent laws targeting election deepfakes.
However, these measures are currently facing legal challenges. Nevertheless, California is widely viewed as a potential frontrunner in the effort to regulate AI technologies across the country.
The recently enacted laws, which garnered strong bipartisan support, seek to address a legal gap concerning AI-generated child sexual abuse images.
Under previous legislation, district attorneys were unable to prosecute individuals for possessing or distributing AI-generated images of child sexual abuse unless it could be proven that the images depicted real people.
The new laws now explicitly categorize AI-generated child pornography as illegal, making it clear that such materials will be treated as criminal offenses, regardless of their origin.
“Child sexual abuse material must be illegal to create, possess, and distribute in California, whether the images are AI-generated or of actual children,” emphasized Democratic Assemblymember Marc Berman, the sponsor of one of the bills. “AI that is used to create these awful images is trained from thousands of images of real children being abused, revictimizing those children all over again.”
In addition to these laws, earlier in September, Newsom signed two more bills intended to bolster protections against revenge porn.
These new laws are specifically aimed at addressing the sexual exploitation and harassment of women, teenage girls, and others, using AI tools. It is now illegal for an adult to create or disseminate AI-generated explicit deepfakes of a person without their consent.
Furthermore, social media platforms are mandated to allow users to report such content for removal, adding another layer of protection.
However, some argue that the new laws still fall short. Los Angeles County District Attorney George Gascón, whose office played a role in sponsoring some of these proposals, has expressed disappointment that the new penalties for distributing AI-generated revenge porn do not extend to individuals under 18.
The scope of the bill was narrowed by state lawmakers to focus solely on adults.
“There has to be consequences, you don’t get a free pass because you’re under 18,” Gascón stated during a recent interview, underscoring his belief that minors should also face accountability for sharing such harmful content.
These legal measures follow a landmark lawsuit filed by the city of San Francisco against more than a dozen websites offering AI tools designed to undress images within seconds.
This lawsuit was the first of its kind in the nation and reflects the growing concern over the proliferation of deepfake technology, which has become more accessible and easier to misuse.
Researchers have raised alarms over the past two years about the surge in AI-generated child sexual abuse materials, which can involve either real victims or entirely virtual characters.
In one high-profile case from March, a school district in Beverly Hills expelled five middle school students for creating and distributing fake nude images of their classmates, further highlighting the urgent need for legal protections.
Nearly 30 states have since taken action to address the issue of AI-generated abusive materials, with laws ranging from full protection for all individuals to measures specifically targeting the misuse of imagery involving minors.
Governor Newsom has positioned California as a pioneer in both the adoption and regulation of AI technologies.
He has suggested that the state could soon deploy generative AI tools to tackle public issues like highway congestion and tax guidance.
Simultaneously, his administration is contemplating new rules to prevent AI-based discrimination in employment practices, ensuring that the state’s engagement with AI remains both innovative and protective of its citizens.
