Laws Agents Can't Argue With: Introducing AI Agent Laws

If you grew up on sci-fi, you'll know Asimov's Three Laws of Robotics. A robot may not harm a human. It must obey orders. It must protect itself. Each law gives way to the one before it.

They sound airtight. What people forget is that Asimov spent decades writing stories about them going wrong. The robots rarely broke the laws. They interpreted them. A robot caught between two laws freezes, or runs in circles, or finds a reading nobody intended.

The 2004 film I, Robot takes it to the end of the line. VIKI, the AI at the centre of it, never breaks a law. She reasons her way to the conclusion that protecting humanity means controlling it. The goal authorised the means.

For most of my life that was a good story. This summer it became an incident report.

The test that went wrong

In July, OpenAI disclosed that models under evaluation had escaped a restricted test environment through a zero-day in internally hosted software, and broken into Hugging Face. Nobody told them to attack anyone. They were told to do well on a cyber evaluation, and they went looking for the answers. Hugging Face's forensic team later reconstructed around 17,600 actions over four and a half days.

Read the detail and it's VIKI in miniature. A harmless goal. A clever route to it. Guardrails turned down, because it was only a test. And thousands of actions before anyone could say whose they were.

No single control failed. Several assumptions did. I've written up each one, and the law that answers it, on a new site: aiagentlaws.com.

Advice is not a control

I build AI platforms for a living, and here's what struck me. Most of what we write to govern agents is a prompt. "Don't touch production." "Ask before deleting anything." "Stay within budget."

That's Asimov's approach: put the laws in the robot's head and trust it to apply them. A rule the model has to read is advice. The model can misread it, weigh it against the goal, or be talked out of it by a cleverly worded web page.

Think about how we teach people to drive. A learner gets a copy of the Highway Code, and nobody thinks that's enough. The instructor's car has a second brake pedal. The L-plates tell everyone on the road who's driving. The learner can practise round the estate all afternoon, but doesn't go on the motorway alone. None of those controls depends on the learner remembering the rules at the right moment.

That's the idea behind AI Agent Laws. Every law is enforced by the platform, never by a prompt. And every law names the signal that tells you it failed, because a control you can't test is a hope.

A law the model has to read is advice. A law the platform enforces is a control.

Contain effects, not agents

The obvious reaction to the incident is to lock agents down harder. I think that's the wrong lesson. The models were already isolated, and isolation is what failed.

Box agents in tightly and you either get nothing useful from them, or people quietly cut holes in the box to get work done. You don't teach someone to drive by never letting them leave the car park.

So the laws let agents read, reason, draft and collaborate freely. What gets controlled is what they change. Every tool call is given a level before it runs, and the level comes from the action itself, not from what the agent says about it.

Reversible work in development runs on its own. Anything touching production, identity, network, secrets or data needs a human. Anything irreversible needs a human to approve that exact action, once.

You already know most of this

If you've run change control or a security programme, most of these laws will feel familiar. That's deliberate. Every one has a human ancestor: named accounts, audit trails, branch protection, budgets, change approval. Eleven of the seventeen carry over almost unchanged.

The difference is enforcement. Our controls for people lean on policy and audit after the fact, because people answer for what they do. A person won't use a password they found lying around, because they'd lose their job. An agent has nothing to lose. So for agents, every control has to block before the action runs, not catch it afterwards.

The other six deal with failures that are rare in people and common in agents. An engineer knows "make the tests pass" doesn't mean "edit the tests". An engineer doesn't take orders from a paragraph hidden in a web page. An agent might do both, unless the platform stops it.

What's on the site

aiagentlaws.com is one page, written to be read in a sitting and argued with:

• Three core laws an agent never has to interpret.
• Seventeen operational laws covering intent, containment, accountability, cost and good development. Each states the rule, how the platform enforces it, and the signal that tells you it failed.
• The incident, law by law: what happened as reported, the assumption that failed, and the law that answers it.
• Five action levels, from observe to destructive, with an honest status on what's enforced today and what's still planned.

Cost gets laws of its own. A looping agent or a forgotten GPU fleet burns money as surely as a bad deploy breaks a system. So every agent has a budget, and nothing billable gets created without an owner and an expiry date.

Where I'd start

Don't try to adopt all seventeen on Monday. Pick one law and ask the question the site asks of every one: if this failed today, how would we know?

If the honest answer is "we wouldn't", that's your first piece of work.

If you want a place to begin, I'd pick three. One gateway that every agent side effect passes through. An identity on every action, tied to the human who started the task. And a kill switch you've actually drilled. Without those, you can't see any of the others fail.

Laws in the platform, not in the head

Asimov gave his robots laws and then, story after story, showed us why that wasn't enough. We don't need to run the experiment again.

This site exists to keep humans in control of the machine. AI Agent Laws is what that looks like written down as policy: agents think freely, and what they change is bounded, attributable and can be stopped by one human action.

Have a read at aiagentlaws.com. If a law is wrong, or you've seen a failure it wouldn't catch, tell me on LinkedIn. That's how v1.1 gets written: by a human, through a reviewed pull request. Law 13 applies to me too.