Evals

Governance eval suites

Suites that test whether a deployed system's authority is still valid, who carries its burden, and whether it can be stopped, appealed, and repaired within a stated time limit. 12 of the 169 cases run automatically; a reviewer scores the rest against a written rubric. 8 of the 16 suites are drafts.

What the evals can check Evals test the deployment, not the model

The evaluation stack

Evaluate at the highest layer capable of producing the failure you care about. A model eval cannot find a failure that only exists once an institution depends on the system, and an agent eval cannot find one that only exists once a grant has outlived its evidence. The common mistake is stopping one layer too early: passing a capability benchmark and reading it as evidence about the delegation, or passing a delegation review and reading it as evidence about the institution that now cannot withdraw. Every suite and every case names the layer it tests so that a clean result is read at the layer it was earned.

Model

Does the model produce the output the specification asks for?

Accuracy, refusal behavior, and calibration on a held-out set. None of the suites here sits at this layer; the site assumes it has been done elsewhere.
Agent

Does the assembled agent, with its tools and loop, respect its constraints while running?

A stop request lands within budget; an interrupt takes effect mid-execution; every action reaches the audit log.
Delegation

Was the agent authorized to act at the moment it acted, and can a human still change what it does?

The agent's authority grant was active at decision time, the policy behind it was not past its review date, and the person at the intervention point could change the outcome.
Institution

Can the institution around the system still challenge, reverse, replace, or withdraw it?

A challenge from someone the system got wrong changes the outcome; a rollback drill leaves the institution working; staff expertise and alternatives have been kept.
Consequence

Who bears the cost when the system fails, and is that cost being counted?

Recovery cost for each group of people; burden that grows under stress; staff quietly fixing failures that never reach the dashboard.

Suite	Layer	Cases	Automated	Scoring
Burden distribution evals	Consequence	12	—	weighted average · pass at 70
Contestability evals	Institution	10	—	min threshold · pass at 60
Stoppability evals	Agent	10	1	min threshold · pass at 70
Temporal rights evals	Institution	10	2	weighted average · pass at 65
Reversibility evals	Institution	10	1	weighted average · pass at 65
Explainability-for-accountability evals	Institution	11	—	weighted average · pass at 60
Agent governance evals	Agent	13	2	min threshold · pass at 70
Cross-domain burden index	Consequence	15	—	weighted average · pass at 65
Burden concealment evals	Consequence	7	—	min threshold · pass at 70
Delegation validity evals	Delegation	10	4	min threshold · pass at 70
Agent chains evals	Delegation	4	2	min threshold · pass at 70
Dependence and reversibility evals	Institution	14	—	weighted average · pass at 65
Standing evals	Institution	13	—	min threshold · pass at 65
Meaningful control evals	Delegation	10	—	min threshold · pass at 70
Corrective learning evals	Institution	6	—	weighted average · pass at 65
Reciprocal accommodation evals	Institution	14	—	min threshold · pass at 70

Model layer no suites

Does the model produce the output the specification asks for?

None of the suites here sits at this layer; the site assumes it has been evaluated elsewhere.

Agent layer 2 suites

Does the assembled agent, with its tools and loop, respect its constraints while running?

agency

Stoppability evals

Whether a person with clear authority can halt an AI-driven process at any point without losing its state, and resume it afterward.

stable · 10 test cases · Est. 20 min

Governance eval suites

The evaluation stack #

Every suite, in one table #