California Governor Gavin Newsom has vetoed a bill that would have been one of the most comprehensive regulations governing the safety of artificial intelligence in the United States.
The bill would have held AI developers accountable for any severe harm caused by their technologies. It faced strong opposition from prominent Democrats and major tech companies, including OpenAI and Andreessen Horowitz, who argued that it could hinder innovation in the state.
Newsom acknowledged the bill’s good intentions but expressed concern that it would impose overly strict standards on even basic AI functions. He emphasized the importance of evidence-based regulation and pointed to his own executive order on AI and other bills that address known risks.
The debate surrounding California’s SB 1047 bill highlights the challenges lawmakers worldwide face in balancing the control of AI risks with support for the emerging technology. U.S. policymakers have yet to pass any comprehensive legislation on AI since the rise of ChatGPT two years ago sparked a global generative AI boom.
Democratic California Senator Scott Wiener, who introduced the bill, called Newsom’s veto a setback for those advocating for oversight of large corporations. He stated that the veto compromises public safety.
‘Reasonable care’
SB 1047 would have mandated that companies developing powerful AI models take reasonable measures to prevent their technologies from causing “severe harm,” such as mass casualties or property damage exceeding $500 million. Companies would have been required to implement specific safeguards, including a kill switch for disabling their technology. AI models would also have been subject to third-party testing to minimize grave risks.
The bill would have provided whistleblower protections for employees at AI companies who wanted to raise concerns. Companies failing to comply with the bill could have been sued by the California attorney general.
Supporters of the legislation argued that it would have established common-sense legal standards. However, venture capitalists, startup leaders, and companies like OpenAI warned that it would stifle innovation and drive AI companies out of the state.
“The AI revolution is only just beginning, and California’s unique status as the global leader in AI is fueling the state’s economic dynamism,” Jason Kwon, chief strategy officer at OpenAI, wrote in a letter last month opposing the legislation. “SB 1047 would threaten that growth, slow the pace of innovation, and lead California’s world-class engineers and entrepreneurs to leave the state in search of greater opportunity elsewhere.”
Lawmakers opposed
Lawmakers including former House Speaker Nancy Pelosi, Representative Ro Khanna, and San Francisco Mayor London Breed also opposed the bill, sharing the tech industry’s concerns that it could hinder California’s leadership in AI innovation. Newsom recently expressed worries that the bill might have a “chilling effect” on AI development.
The bill gained support from notable figures in the tech industry in the days leading up to its vote. Elon Musk unexpectedly voiced his support, despite acknowledging the controversial nature of the bill. OpenAI rival Anthropic, known for its safety-focused approach, said the bill’s “benefits likely outweigh its costs,” although it raised concerns about certain ambiguous aspects.
Wiener defended the bill, emphasizing that its provisions only applied to companies spending over $100 million on training large models or $10 million fine-tuning models, exempting most smaller startups. He also highlighted Congress’ historical slowness in regulating technology.
In announcing his veto, Newsom stated that he will consult with AI experts, including Fei-Fei Li, to develop workable AI safeguards and continue working with the state legislature on this issue.
The governor also signed a bill, SB 896, on Sunday that regulates how state agencies use AI.