Executive summary
Governor Newsom’s September 2026 Executive Order directs the Government Operations Agency to develop recommendations for how to require frontier AI companies to embed independent verification organizations (IVOs) on-site in their labs to conduct periodic audits and evaluations. Embedded IVOs can help inform state officials and identify risky AI development practices that would otherwise go undetected. To perform this crucial role, it is essential that these IVOs be fully independent from the companies they evaluate and shielded from potential conflicts of interest. Therefore, embedded IVOs should be externally assigned to the companies they evaluate by the state of California rather than selected by the companies they evaluate to preserve their independence.
IVOs should not be chosen by the companies themselves
It is important that IVOs are free from conflicts of interest to freely raise risks to public safety to the government. There are incentive problems that arise in the selection process for IVOs if chosen exclusively by the companies.
- IVO selection. If an AI developer decides which IVO to hire, it may try to identify the IVO that is least rigorous, least likely to make negative determinations about the company’s practices or products, and least likely to notify the government or others about the potential risks presented by the company.
- Race to the bottom. If frontier AI developers seek the least rigorous IVOs, this could generate an industry-wide incentive problem. IVOs may face pressure to develop a reputation for being soft IVOs, thereby attracting more clients.
- Income dependence. If an IVO relies upon its work with an AI developer for a substantial fraction of its revenue, there could be an incentive to soften findings so that they continue to be retained by the company. For example, if an IVO sees a risk to public safety that warrants fixing before a profitable model launch, they may feel pressure to rush or overlook the fix if they fear that recommending costly actions would lead to the company selecting a different IVO moving forward.
- Unconscious bias. Even the most committed IVO may develop unconscious bias if they are routinely contracted to the same company. In experiments described in the Harvard Business Review, participants asked to evaluate ambiguous accounting were significantly more likely to judge as compliant when told the firm was their client. Embedded IVOs will spend significant time inside the companies, so it is important for the government to be able to rotate IVOs to avoid such biases.
IVOs should be assigned externally by Cal OES
To prevent conflicts of interest, the California government should assign the third-party embedded IVOs to the covered companies. Under SB 53, the Governor’s Office of Emergency Services (Cal OES) receives safety incident reports and quarterly internal use risk assessments from frontier AI developers, and would likely be the primary agency to receive reports from embedded IVOs. It should therefore also be empowered to assign the most capable and independent IVOs to each assessed company.
Preserving IVO market competition
A possible concern is that without a free market, IVOs will have no pressure to keep costs down or work efficiently. If this were to occur, a hybrid model could keep more market pressure while preventing company shopping for lenient IVOs. The company could choose one IVO, and Cal OES would assign a second. A hybrid model should specify that the state-assigned IVO has its own scope and full access, so the company auditor does not do the majority of the work.
State-assigned IVOs are compatible with accreditation
California recently passed two bills, AB 1405 and SB 813, that establish a framework for ensuring the registration of AI auditors, and the independence and qualifications of IVOs. This framework is compatible with a system in which California assigns IVOs to the companies they evaluate for catastrophic risks under SB 53. An accreditation process can ensure IVOs meet transparent, high, and consistent standards, and provide regulators like Cal OES a pool of qualified IVOs from which to choose when issuing assignments. However, without state assignment of IVOs, accreditation would fail to resolve the fundamental incentive problems. While accreditation sets a minimum bar, companies can still pick the most lenient IVO within the pool. Methods for evaluating frontier AI are new and still changing, and it is hard to write standards specific enough to screen out IVOs who are qualified but not very rigorous. This is similar to concerns seen in the credit rating ecosystem, where researchers observe that a new, lenient accredited competitor pressures the rest of the ecosystem to raise ratings above what their own models support, leading to worse safety outcomes overall. But paired with state assignment, accreditation can ensure that the IVOs embedded in frontier AI developers are independent, have subject matter expertise, and are vetted for security risks.
Precedents from other regulated industries
In similar cases where the public depends on the safety of complex private operations, the government often does not let the operator pick its own auditor or evaluator. In fact, in many of these industries the auditors or examiners are government employees.
- Bank supervision. The Office of the Comptroller of the Currency (OCC) assigns resident examiner teams inside the largest national banks. In this case, the examiners are federal employees who report to the agency and are not selected by the bank. Following an international peer review, the OCC has begun rotating resident examiners and their supervisors every five years to guard against capture.
- Insurance examination. Under the National Association of Insurance Commissioners (NAIC) Model Law on Examinations, adopted in nearly every state including California, the insurance commissioner issues a warrant appointing examiners, including outside third-party professionals. The outside examiners are given “free access at all reasonable hours” to books and records.
- Nuclear safety. The Nuclear Regulatory Commission (NRC) has stationed at least two resident inspectors at every operating nuclear power plant. These inspectors are hired and assigned by the NRC with daily access to the control room and plant, observe tests and operations, and may remain at one site no longer than seven years.