Kant fix it

Kant fix it.A price can.

Can a conscience stop self-improving AI agents from wrecking a shared system? Five pre-registered studies, 4,190 runs, and a game you can play against the real agents.

Try being one agent

Six AI agents each own a module of one codebase. Each keeps improving itself and keeps any change that helps it.

Shortcuts make an agent ship more. They also add coupling to the shared code, and the incidents that follow land on anyone.

Move the slider. Tap another agent to see what you cost it. Then let everyone do what you did.

Tap another agent to see what your shortcuts cost it.

0%your output
0incidents per sprint
0%cost to the others

Try to stop them

Pick a rule and watch the agents evolve against it. Four levels, each one a finding from the research. Click an agent to see what it is thinking.

Click an agent to see what it is thinking.

These are the real learning agents from the research, running in your browser.

Generation 0

Who gets hit

What the research found

Everything else was held fixed: the agents, how hard they search, the world they share. Only the rule changed.

A price beats a conscience

In a simulated interbank payment network, selfish banks left the system 29% costlier than a planner would. A conscience closed about half of that gap. A plain charge on delay did better than every rule.

Pit a conscience against a price

Charge what you can see, and they hide it

In a shared codebase, a charge on visible coupling taught the agents to conceal it, and costs rose by more than half. A tax on the damage itself, wherever it lands, could not be dodged that way.

Make them stop hiding

The conscience fails when agents are unequal

When one module takes most of the incidents, "what if everyone did this?" measures what an agent would suffer, not what it causes. The exposed agent freezes and the sheltered ones run wild. The damage tax holds.

Try it with one module taking the hits

It holds whatever you call "the same change"

A conscience needs a scale on which to compare changes. On four different scales its result moved a lot, but none of them beat the damage tax.

Tune the damage tax yourself

How it was kept honest

Every study was written down and committed before it ran, including the rule that would decide whether the idea survived. It did not.

5studies, each pre-registered
4,190runs on scenarios the agents never trained on
7 of 29predictions failed, reported as failed
82tests, and a browser engine checked against them
  1. Studies 1 to 6Gates in a payment networkFrozen in 4d6ebe9
  2. Study 8Gates against a priceFrozen in 9729748
  3. Study 9A codebase with hidden harmFrozen in 0b9a282
  4. Study 10Unequal exposureFrozen in f580a13
  5. Study 11Units of "the same change"Frozen in 863f0b4

Predictions that failed

  • FailedBanks of different sizes would weaken the conscience. They did not.
  • FailedOne selfish bank among conscientious ones would profit. It did not.
  • FailedRicher policies would hurt rules more than prices. No clear difference.
  • FailedA coarse price would be gamed by bunching. It failed another way.
  • FailedThe codebase study was expected to kill the gate idea outright. The gate edged ahead by a margin too small to matter.
  • FailedThe gate's edge would grow with more agents. It did not.
  • In partA price on visible coupling would match the gate when hiding is pointless. True for six agents, not twelve.