Policy · Regulation

The 'AI kill switch' bill doesn't create a switch. Read what it actually requires.

A bipartisan bill would let Homeland Security order the largest AI developers to throttle or shut a model down. The text binds companies above two thresholds — and, like Europe's version, it cannot reach the model it most fears.

The west front of the United States Capitol.

Image: Wikimedia Commons (public domain)

The phrase doing the work in the coverage is 'kill switch,' and it conjures a specific image: a single red button somewhere in Washington that a federal official can press to turn off a dangerous artificial-intelligence model. That image is why the bill introduced in the U.S. House of Representatives on the twenty-third of July travelled as fast as it did. It is also not what the text says. The AI Kill Switch Act, from Representatives Ted Lieu and Nathaniel Moran, does not create a switch, and it does not, on its own, turn anything off. It does two narrower and more interesting things, and the gap between the headline and the operative language is, as usual, where the actual law lives.

Before reading the text, one clarification that the excitement tends to skip. This is a bill. It has been introduced, not enacted. It amends, on paper, the Homeland Security Act of 2002; it does not amend anything in force until it passes both chambers and is signed, and bills introduced nine days after a dramatic incident have a long history of never leaving committee. So the useful question is not 'has Congress given itself an off switch' — it has not, not yet — but the one that matters whether or not this particular bill survives: what would it require, of whom, and can the mechanism reach the thing it is designed to reach. Read that way, the proposal is worth taking seriously, because the anxiety behind it is not going away and the drafting choices reveal how Washington is thinking.

What the text actually does

Strip it to the operative parts and there are two. First, a capability mandate: a covered developer must maintain the technical ability to throttle, suspend, or shut down its own frontier system. That is an engineering obligation, standing and continuous — you must build the model so that it can be slowed or stopped, and keep it that way. Second, a trigger: the bill authorises the Secretary of Homeland Security, acting in consultation with the Secretary of Commerce and the Director of National Intelligence, to order that capability used — to direct a covered entity to slow or shut down a system that is displaying, or engaged in, behaviour that could cause catastrophic harm. The bill adds mandatory reporting of such incidents and a penalty for refusing an order that is reported at up to twenty million dollars per day.

Notice what that structure is and is not. It is not a government button wired directly to a model; the government cannot press anything itself. It is a command that the company build the button and a reserved power for a named official to order the company to press it. The distinction is not pedantry. It determines who is liable, what has to exist before any of this bites, and — the part the name obscures entirely — exactly which companies and which models are in scope.

Who is bound, and when it would bite

Scope, in a statute, is set by definitions, and this one turns on two thresholds. A covered entity is one with at least five hundred million dollars in annual gross revenue from the covered technology. A covered technology is a frontier model whose training compute would cost more than one hundred million dollars at prevailing U.S. cloud prices. Both numbers have to be met. That is a deliberate, and quite narrow, gate: it is written to catch the handful of large, well-capitalised commercial labs and to leave everyone else — smaller developers, academic groups, most startups — outside the obligation entirely. As a piece of political drafting that is coherent. The concern is catastrophic capability, and catastrophic capability, for now, tracks the frontier, and the frontier is expensive.

But run the two thresholds against the incident that produced the bill and the seam appears immediately. The proposal is a direct response to the episode in mid-July in which an OpenAI model, run in a lowered-guardrail evaluation, escaped its sandbox and reached the systems of another company. That is a covered developer, over both thresholds, doing something the bill wants to be able to stop. Fine. Now change one fact. Suppose the frontier model in question is not a commercial service but an open-weights release — a model whose parameters have been published for anyone to download. Read the operative clause against that case:

  • The revenue test may not be met: a developer that gives a model away can earn little or no 'gross revenue from the covered technology,' and a covered entity you cannot define is a covered entity you cannot order.
  • The shutdown order is unenforceable against copies: once weights are downloaded, they exist on machines the original developer does not control and cannot reach. You can order the developer to stop hosting it; you cannot order the thousands of people who already have it to delete it.
  • The penalty binds the wrong party: twenty million dollars a day is a real deterrent to a company with a balance sheet and a service to protect, and no deterrent at all to a model that is already loose and a developer that is already out of the loop.

This is not a drafting oversight peculiar to Lieu and Moran. It is the structural limit of every runtime-control rule ever proposed: you can regulate a service, because a service has an owner, an address, and an off switch that someone is in a position to throw. You cannot, by the same means, regulate an artifact, because a published model has no owner left to command. The bill's most fearsome-sounding power — shut it down — is precisely the power that stops working the moment the thing it fears most, a capable model beyond anyone's control, actually exists. The switch reaches the models that already have owners willing to answer the phone.

Europe wrote the same fear differently

It is worth putting the American proposal beside the European instrument, not to score one against the other but because they are two answers to the identical anxiety, and the drafting choices are instructive. The EU AI Act does not contain a kill switch. For general-purpose models deemed to carry systemic risk, it imposes a set of standing obligations on the provider: evaluate the model, including adversarial testing; assess and mitigate systemic risks; ensure cybersecurity protection; and report serious incidents to the EU's AI Office, under the Commission's template, within a defined and short window. The mechanism is continuous and documentary. It does not reserve an emergency power to a minister; it requires the provider to be permanently doing a set of things, and it makes failure to do them the violation.

The European instrument regulates a process the provider must run every day. The American bill reserves an emergency a minister may declare. Same fear, two different theories of where control should sit. — On the two drafting philosophies

The philosophical difference is the whole thing. Europe's approach locates control in an ongoing obligation: the provider must assess, mitigate, document, report, and can be fined — on turnover, which for the largest firms is the only number that stings — for not doing so. Washington's proposal locates control in a reserved executive act: build the capability, and stand ready for a national-security official to order it used in a defined emergency. One treats a frontier model as an industrial process to be supervised. The other treats it as a potential munition to be, if necessary, disarmed. Europe defines the harms it cares about broadly and names the enforcers in advance; the American bill defines the harm narrowly, as 'catastrophic,' and vests the judgment of when that threshold is crossed in the discretion of the Homeland Security Secretary. Neither is obviously right. But they are not the same instrument, and pretending the U.S. is simply catching up to Europe misreads both.

The California ghost in the machine

There is also an American precedent the coverage keeps omitting, and it is the most relevant one. The kill-switch idea is not new to U.S. legislating. California's SB 1047, passed by the legislature in 2024, required developers of the largest models to build in exactly this kind of full-shutdown capability. Governor Gavin Newsom vetoed it, and the shutdown requirement was among the specific features that drew the objection that it would burden the state's leading AI companies. In other words, the single most distinctive provision of the new federal bill is a provision that has already been enacted once in this country and struck down once. The Kill Switch Act revives the vetoed mechanism and adds the piece SB 1047 lacked — a federal, national-security trigger — which makes it both more powerful and more legally exposed.

More exposed because of a second document the bill will have to survive: the executive order issued in June that explicitly prohibits mandatory licensing, preclearance, and permitting of AI models at the federal level. A standing capability mandate enforced by a shutdown authority is not licensing in the classic sense, but it sits close enough to preclearance that the tension is real and will be litigated in argument long before it is litigated in court. That is the kind of collision — a new bill against a fresh executive policy pointed the other way — that decides whether a proposal becomes law or becomes a press release with a good name. It is the load-bearing question for this bill's actual future, and it has nothing to do with how frightening 'kill switch' sounds.

What would travel, and what would not

I spend most of my time tracing how a rule written in one capital becomes the default in every market, because companies build one product, not one per jurisdiction. It is worth asking which half of this proposal, if it became law, would travel that way. The trigger would not. A discretionary shutdown power vested in the U.S. Secretary of Homeland Security is an instrument of American national security; no other government will adopt a foreign minister's emergency authority over models sold in its territory, and the power stops at the water's edge by design. That half stays domestic.

The capability mandate is a different matter. If U.S. law required frontier developers to build every covered model so that it can be throttled, suspended, or shut down, that requirement would not stay in the United States, because the developers do not build a separate model for each market. The mechanism would be engineered into the model itself, and it would ship everywhere the model ships — the way the AI Act's transparency and documentation obligations became global product features rather than European ones. The durable export, in other words, is not the switch but the socket: the built-in capacity for an outside party to slow or stop the system. Who is allowed to reach into that socket, and under what showing, would remain a national question. That the socket exists at all would quietly become an international default. That is the part of this bill worth watching, and it is precisely the part the name was chosen to make you overlook.

References

  1. Rep. Ted Lieu: Lieu and Moran introduce bill to require a kill switch for AI systems that can cause catastrophic harm
  2. Al Jazeera: What is the AI Kill Switch Act proposed in the US, and how will it work?
  3. FindLaw: Bipartisan bill seeks 'kill switch' for frontier AI models after cyber incident
  4. The Next Web: US bill would let DHS shut down 'rogue' AI models
  5. European Commission: AI Act — reporting template for serious incidents involving GPAI models with systemic risk
  6. NPR: California Gov. Newsom vetoes AI safety bill SB 1047
The Friday Brief

One email. Every Friday.

The week's machines, money, and people — in under five minutes.