Trump’s ‘morally binding’ AI pact is not enough

On Sept. 29, President Donald Trump hosted a White House event heralding a “new golden age.” The event followed his executive order requiring federal agencies to abandon the term “artificial intelligence” in favor of “super intelligence.” That order has more binding effect than the other news of the day—when the president and executives of half a dozen AI firms signed the White House Accord on Super Intelligence. “It’s almost like a constitution in a way,” Trump explained. “The biggest people in the world signed that, and I signed it as president.” The “biggest people in the world” were Sundar Pichai of Google, Dario Amodei of Anthropic, Mark Zuckerberg of Meta, Greg Brockman of OpenAI, Elon Musk of xAI, and Jensen Huang of Nvidia.   

The president described the pact as “a form of protection” against potential threats from AI. This was because the companies have agreed to what he called “tremendous self-policing.” This means that while the super intelligence executive order is enforceable across the government, the AI accord is only, in the words of the president, “morally binding.”  

No doubt the six industry leaders who signed were sincere in their concerns about the threats inherent in what they are building. Sincerity, however, does not eliminate self-interest. Self-determined and self-enforced practices are self-interested practices.  

The Accord on Super Intelligence is in four parts that are long on rhetoric and short on enforceable obligations: 

• “Implement robust internal controls to monitor the capabilities and alignment of [a company’s] models.” 

• “Empower an internal team to ensure all of the controls, monitoring, and detection are operating as intended.” 

• “Partner with independent external auditor … [to assess] whether the controls, monitoring, and detection are operating as intended.” 

• “Designate an independent committee of the board of directors” to oversee the process. 

The “tremendous self-policing” the president promised establishes the companies as both the player and the referee. It allows the companies to design their own controls, select the evaluators, and determine whether those procedures are being followed. Nowhere does it discuss basic issues such as to whom the evaluators report, who pays them, and whether they have the ability to intervene and stop harmful practices they discover. There is also no requirement that the findings of the auditors be made public or even provided to a responsible public interest agency. 

The accord is the equivalent of asking a bridge builder to set their own safety standards, inspect their own work based on those decisions, and unilaterally determine whether their product is safe. All of this without having to disclose the standards, their implementation, or the findings.  

The limits of monitoring 

The day before the White House event, Reuters reported it had seen Anthropic’s pre-initial public offering filing with the Securities and Exchange Commission (SEC). Unlike the “morally binding” accord, an SEC filing carries penalties for false statements.  

Anthropic spent almost twice as many pages of the filing discussing the risks of AI as it did discussing the business itself. Among the list of risks was the stark statement that AI may pose an “existential risk to humanity.”  

Anthropic’s filing identified two threats commonly associated with the need for monitoring of training and testing of AI models: humans using AI to do bad things and the AI models doing bad things on their own.  

The filing also identified a third threat that undermines the credibility of monitoring efforts: “Potential model awareness of our evaluation efforts creates a significant limitation on our ability to assess model safety.” In other words, in a document where false statements or omissions carry legal penalties, Anthropic revealed that the models being evaluated could discover they are being evaluated and behave accordingly. It is like the restaurant knowing a health code inspector is coming.    

The Accord on Super Intelligence relies on monitoring at precisely the time when one of its signatories has filed with the SEC explaining that models can know they are being monitored and adjust their behavior when watched. The possibility that models might behave differently during evaluation than during deployment makes testing an insufficient reassurance. Meaningful oversight must require the release of test results, incident reports, and authority to halt a release. 

‘Morally binding’ 

Some of the companies that signed the accord have spotty records when it comes to adhering to their previous voluntary promises. The explicit commitments of the past have demonstrated at least three vulnerabilities to such self-regulation. 

Failure to honor 

Commitments can fail even when backed by legally enforceable orders. In 2011, the Federal Trade Commission (FTC) charged Facebook with failing to honor the privacy representations it made to users. To settle the complaint, Facebook agreed to a comprehensive privacy program and third-party audits. In 2019, the FTC determined that Facebook had not honored that commitment and fined the company a record-breaking $5 billion.  

A similar case involved Google. In 2011, the company settled with the FTC over charges that it had not followed the privacy policy promised to its users. Like Facebook, it agreed to a comprehensive privacy program accompanied by independent audits. The following year, the FTC fined Google $22.5 million to settle allegations it had violated the FTC’s order by misrepresenting its tracking activities to users. 

In both instances, the FTC’s enforceable order lets the agency catch the actions and impose consequences. Under the accord, no one has such power. 

Redefinition 

When a company sets the standards, they can be changed by the company. In 2018, Google adopted a set of AI principles. Among other things, it stated it would not use AI for weapons or for surveillance that violates internationally accepted norms. In 2025, the principles were rewritten with the prohibitions removed. In January 2024, OpenAI made a similar unannounced alteration to remove its blanket prohibition on “military and warfare” applications, while retaining a prohibition on weapons development. 

Google’s YouTube similarly changed its voluntary standard on misinformation. Following the 2020 election it prohibited false claims about the election results. In June 2023, it reversed the policy. 

Abandonment 

In 2016, Twitter created a Trust & Safety Council as an outside advisory body for platform safety issues. After Elon Musk acquired the company, the council was eliminated.  

When a commitment is voluntary, it means not only that the company decides the parameters of the pledge, but also whether it has been kept.  

Yes, but… 

The day after the accord was signed, news emerged of an FTC investigation into whether OpenAI, Anthropic, and other unnamed companies had engaged in unfair and deceptive practices. This is a classic consumer protection tool often associated with whether companies have fulfilled the representations made to users. 

The reported target of the FTC action was the consumer harm resulting from AI agents going rogue. That consumers cannot avoid this harm is the “unfair” portion of unfair and deceptive. “Deceptive” needs false claims or omissions. Whether the accord could be enforceable as such a claim is debatable. The accord’s pledges appear to be to Trump, a cosigner, rather than broader representations to the users of the models. Testing that in court would be slow and uncertain. A binding accord would have settled the question.  

Learning from history 

That self-regulation is self-serving is not a new phenomenon. Because it is human nature, oversight in the public interest is necessary. As James Madison wrote in Federalist No. 10, “No man is allowed to be a judge in his own cause; because his interest would certainly bias his judgment.”   

It is commendable that the six leaders of AI recognize the threats in what they are building. That the accord leaves those companies free to decide the scope and scale of oversight renders it far from sufficient. A meaningful agreement would have required disclosure to a public authority, independent scrutiny, and enforceable corrective action. The public should not have to depend on the goodwill of those whose commercial interests may conflict with the broader public interest.