Ethotechnics Institute
The appeals kept winning. The system kept running.
Australia's Robodebt raised 470,000 unlawful debts over three years. A tribunal ruled individual debts unlawful dozens of times, and each ruling fixed one person's debt. Nobody with the power to stop the scheme treated those rulings as evidence against it.
We publish free, open standards for one question: when an automated system is hurting people, can anyone stop it?
Something going wrong right now? Go to incident triage →
- 2014 The department is advised in writing that income averaging cannot prove a debt.
- Jul 2016 The automated scheme launches. Debt notices go from about 20,000 a year to 20,000 a week.
- Apr 2017 The Ombudsman reports that notices do not explain how a debt was calculated.
- 2017–19 A tribunal rules individual debts unlawful dozens of times. Each ruling fixes one case. The scheme keeps running.
- Nov 2019 The government concedes a Federal Court case it was about to lose. The scheme is halted that month.
- Jul 2023 A Royal Commission finds the scheme was unlawful from the outset.
60-second self-test
Could anyone stop yours?
Pick one system that decides something about people: a loan, a shift, a benefit, a refund. Answer six questions about it. "Not sure" counts as no.
Casebook
Five public failures, scored.
Each was established by a court, an inquiry, or a regulator. In none of them did the organization running the system stop it on its own.
- Robodebt
- The childcare benefits scandal
- Post Office Horizon
- England's 2020 exam grades
- Apple Card credit limits
● held · ◐ drifted · ○ failed. Open the full matrix →
The claim, in one figure
A delegation can double without anyone deciding that it should.
Both grants below grow by the same two percent a month. One is reviewed only when a single step is large enough to notice, so it is never reviewed. The other treats every widening as a fresh authorization, so it can be read, questioned, and reversed. That difference is what the standards on this site are for.
Demonstration
The same growth, decided and undecided
Two grants widen by 2% a month for 3 years. The left one is reviewed when a single step reaches 5%; the right one records every widening as an authorization.
The right-hand grant's state_history, first three entries
of 37
-
{ from: none, to: allowed, reason: issued }scope ×1.00 -
{ from: allowed, to: allowed, reason: expansion }+2%, scope ×1.02, capacity re-checked -
{ from: allowed, to: allowed, reason: expansion }+2%, scope ×1.04, capacity re-checked
A demonstration, not a measurement of any deployment. The steps and the
threshold are in src/utils/ratchet.ts; the transition reason
expansion and the states are the ones
authority-grant.schema.json allows, and a test holds them there.
MEC-19 expansion review reads state_history, and the left-hand
grant gives it nothing to read.
This is Law XI of twelve. Read the laws → See what STD-08 requires →
How it's different
Other frameworks ask whether risk was managed. These standards ask whether the system can be stopped.
They work alongside the EU AI Act, NIST, ISO 42001, and the OECD principles. They cover what those leave vague: who can stop a running system, how fast, and who carries the cost while it runs.
Most AI governance frameworks improve documentation and oversight. They say less about whether a running system can be halted, reversed, and repaired, or how long that takes for the person it is harming.
When a system simplifies its own operations, the work it leaves undone falls on someone — the nurse re-routing a scheduler's misfires, the claimant re-proving a denial. That labor is part of the system, so the person doing it holds standing to correct it (Law VII). The burden should run uphill.
| Existing standard says | Ethotechnics requires |
|---|---|
|
"Maintain human oversight" EU AI Act, Art. 14 |
Named human with stop authority, tested halt path, recovery clock |
|
"Manage risks across the AI lifecycle" NIST AI RMF |
Measurable time-in-harm bounds, exercised rollback and restoration paths |
|
"Conduct conformity assessment" ISO/IEC 42001 |
Evidence that the system can be stopped mid-incident, not just documented as compliant |
|
"Implement responsible AI principles" OECD AI Principles |
Binding escalation: owner + timer + action, or the system degrades |
Each requirement on the right can be tested against a running system. A compliance document cannot pass it on its own. See the full standards comparison for the detailed analysis.
Use and cite this work
Free to use, credit, and adapt
Ethotechnics Institute materials are published under CC BY-SA 4.0 . Credit the Institute, and publish anything you adapt under the same license.
Individual entries and patterns include citation metadata so you can reference exactly what you used.
New and updated
Recent releases
-
Glossary v1.13.0
·
Rewrote category descriptions and scope notes to the current framing, replaced vague or promotional definitions in 89 entries and 109 short definitions, removed unsourced adoption and effect claims, and cut filler words and redundant sentences from 38 entries and 20 short definitions. No term was added, removed, or renamed.
-
Evals v1.6.0
·
Adds seven draft cases for typed decision models: content that claims an approval (AGT-013), the threshold as a policy record (DEL-010), sampling what ran without review (CTL-010), shaping and selecting hops (CHN-003, CHN-004), reasons that belong to the decision (EXP-011), and issue rate against answer capacity (STA-013). 15 eval suites, 155 test cases.
-
Research v1.2.0
·
Marked the three publications as planned studies. Removed sample sizes, findings, and timeline entries that no published data supported, and four bridge artifacts that were never published.
Studio
Need hands-on help?
Ethotechnics Studio provides commissioned support for clinical AI safety evaluation, diligence, governance design, and implementation work.