Contentious California AI Bill Passes Legislature, Awaits Governor’s Signature

By Anna Tong

California lawmakers have passed a contentious artificial intelligence safety bill, setting the stage for one final procedural vote before it reaches Governor Gavin Newsom. He has until September 30 to either sign the bill into law or veto it.

The legislation, designated SB 1047, has generated pushback from tech companies that specialize in generative AI. These firms have expressed concerns that the bill may drive them out of California and stifle innovation. Some Congressional Democrats, including Representative Nancy Pelosi, have also voiced their opposition. On the other hand, proponents of the bill include Tesla CEO Elon Musk, who supports the legislation through his AI firm, xAI.

The proposed measure mandates safety testing for advanced AI models that cost over $100 million to develop or require a specific amount of computing power. Developers must also ensure they have mechanisms in place to deactivate their AI models if necessary, essentially creating a "kill switch."

Furthermore, the bill empowers the state attorney general to take legal action against non-compliant developers, particularly in cases where AI poses ongoing risks, such as the possibility of seizing control of critical government systems like the power grid.

In addition, developers would be required to engage third-party auditors to evaluate their safety practices and implement protections for whistleblowers who report abuses related to AI.

The bill’s author, Democratic State Senator Scott Wiener, represents San Francisco, which is home to companies like OpenAI and numerous startups in the AI sector. He argues that the legislation is crucial for safeguarding public interests as advancements in AI continue to evolve.

Martin Casado, a general partner at venture capital firm Andreessen Horowitz, expressed hope before the vote that Newsom would veto the bill, citing unprecedented bipartisan opposition.

Major tech firms, including Google, Microsoft-backed OpenAI, and Meta Platforms, have raised their concerns in letters to Wiener. Meanwhile, Anthropic, backed by Amazon, indicated that while the benefits of the bill likely outweigh its costs, there are still troubling and unclear aspects that need further consideration.