Definition
Observable system outcomes that prevent harm, distribute burden fairly, and keep people able to challenge decisions and recover. It is judged by measures such as how fast the system halts, whether its actions can be undone, and whether its failures fall hardest on the most vulnerable, not by stated intent. A system does not “behave morally” if those measures are tracked but trigger no halt rules, remediation, or resourcing.
Tests that show it is in place
- Halt, reversal, and challenge measures are tracked and drive operational thresholds or halt rules.
- Post-incident reviews tie outcomes to system controls rather than intent statements.
Hypothetical example
- A payments platform slows risky recommendations, surfaces a human review queue, and documents reversals when fraud indicators spike.
References
- Worked examples · Essay