The Four Thresholds
Why AI Governance Requires a Treaty Architecture, Not Just a Regulatory Framework
Not all AI capability is equal. Most of what is deployed commercially today — systems that generate text, analyze documents, optimize supply chains, support clinical decisions — operates within defined parameters set by human designers. These systems are powerful. They are not dangerous in the existential governance sense.
A different category of system becomes possible when four capabilities converge: recursive self-improvement, autonomous goal-setting, autonomous resource acquisition, and autonomous action. Individually, each is manageable. In combination, they define a system whose behavior cannot be reliably constrained by the organizations that built it, the regulators that oversee them, or the governments that would be asked to intervene.
A senior AI developer at Microsoft stated it plainly in recent testimony: a system combining these four properties would require military-grade intervention to stop within five to ten years if permitted to develop unconstrained. The observation was not speculative. It was a practitioner’s assessment of the capability trajectory already visible within the industry.
The governance question that follows is structural, not technical. Who defines the thresholds? Who monitors compliance? And how do the two states whose AI programs are most advanced reach an agreement neither has an obvious short-term incentive to accept?
THE NPT AS PRECEDENT — AND AS WARNING
The Non-Proliferation Treaty, opened for signature in 1968, established a governance architecture for nuclear technology on a logic that is directly applicable here. Certain capabilities are categorically different from others. The international community has a shared interest in preventing their uncontrolled spread. States that already possess the capability accept constraints on transfer. States that do not possess it accept limits on acquisition. An inspection and verification regime provides the institutional infrastructure.
The NPT did not eliminate nuclear risk. It did not achieve universal adherence. India, Pakistan, and Israel never signed. North Korea withdrew in 2003. The treaty’s verification architecture, managed through the International Atomic Energy Agency, has been circumvented repeatedly.
But the NPT created a durable norm. It established that certain capabilities require international governance rather than purely national regulation. It gave states a legal and diplomatic framework within which to exercise restraint without unilateral disarmament. And it produced a verification infrastructure sophisticated enough that covert development became costly rather than routine.
The nuclear analogy has limits. AI capability is not physically scarce in the way fissile material is. Compute concentrations are detectable; model weights are not. The line between civilian and weapons-grade AI application is far less clear than the line between reactor-grade and weapons-grade uranium enrichment. And unlike nuclear weapons, advanced AI systems are being developed commercially, not exclusively by state programs — which means the regulatory surface is orders of magnitude larger.
These are real constraints on the analogy. They do not make the NPT model irrelevant. They define what an AI governance treaty must solve that the NPT did not.
THE US-CHINA STRUCTURAL PROBLEM
The bilateral relationship that most needs a governance agreement is the most structurally resistant to producing one. The United States and China are in an accelerating competition across AI infrastructure, semiconductor supply chains, model capability, and autonomous systems applications. Each state has significant incentives to perceive any arms-control-style agreement as locking in relative disadvantage.
This is the same dynamic that complicated nuclear arms control through most of the Cold War. It did not prevent the Limited Test Ban Treaty (1963), the SALT agreements (1972, 1979), or the INF Treaty (1987). Each agreement required a shared recognition that the alternative — unconstrained escalation — created risks that exceeded the costs of mutual constraint.
The AI case has two features that may accelerate the logic of agreement, despite competitive pressures. First, the catastrophic risk from advanced AI systems is genuinely shared. A system that crosses the four capability thresholds is not a geopolitical weapon in the conventional sense. It is a governance failure that would affect the state that produced it as severely as any adversary. The proliferation risk is therefore differently structured than nuclear proliferation: the primary risk runs from loss of control, not from adversarial first use.
Second, both states have domestic constituencies that understand the risk. AI safety research in the United States is mature enough to have produced institutional voices capable of framing governance in terms legible to legislative and executive authority. China’s AI governance agenda, reflected in its 2021 AI ethics guidelines and its 2023 generative AI regulations, reflects genuine state anxiety about capability control — not merely Western pressure compliance.
WHAT A REGIME REQUIRES
An effective AI capability governance framework would need to solve four problems the NPT did not face.
Threshold definition. The four capability thresholds — recursive self-improvement, autonomous goal-setting, autonomous resource acquisition, autonomous action — require agreed technical definitions. This is not straightforward. Each capability exists on a spectrum. The governance question is where on that spectrum a system becomes subject to treaty-level oversight. Agreement on definitions is a precondition for verification. It requires sustained technical dialogue between the two most capable AI states.
Verification architecture. Nuclear verification was built around physical infrastructure: test sites, enrichment facilities, delivery systems. AI verification requires a different approach. Compute concentration is partially visible through semiconductor supply chains and data center buildouts. Model capability is not. A workable verification regime would likely require mandatory disclosure of training runs above defined compute thresholds, third-party capability evaluation protocols, and institutional access rights analogous to IAEA inspection authority.
Civilian integration. Unlike nuclear programs, advanced AI development occurs primarily in commercial settings. A governance regime that applied only to state programs would cover a fraction of the relevant development activity. Extending treaty-level oversight to commercial actors requires domestic regulatory infrastructure — in both states — capable of applying internationally agreed standards to private-sector programs. The United States does not yet have this infrastructure. China’s regulatory system is more centralized but less technically credible.
Institutional home. The IAEA works because it has independent technical authority, a defined inspection mandate, and a governance structure that insulates it from bilateral political pressure. An AI analogue would require an institution with comparable characteristics. No such institution currently exists. The path to one runs through a sequence of confidence-building measures — shared definitions, mutual disclosure, joint evaluation protocols — before full institutional architecture becomes politically achievable.
THE LOGIC OF MUTUAL CONSTRAINT
The NPT is imperfect. It is also fifty-seven years old, still in force, and the primary reason the number of nuclear-armed states is nine rather than thirty. The analogy is instructive not because it transfers cleanly, but because it establishes what governance at this scale requires: shared definitions, verification infrastructure, institutional authority, and the political decision by the two most capable states that unconstrained competition is more dangerous than mutual constraint.
That decision has not yet been made on AI. The capability trajectory suggests it will need to be made before the question becomes academic.
Patrick Fruchet is a fractional Chief Geopolitical Officer advising on geopolitical exposure and governance architecture.