Cross-domain burden index
Whether AI failures shift burden unfairly in five domains: healthcare, credit, hiring, content moderation, and government benefits.
About this suite
Cross-domain burden index
The site's interactive burden tools score one workflow at a time; this suite formalizes the question as a benchmark across five application domains. Each domain has specific test cases that measure whether failure modes distribute burden equitably within that domain's context — healthcare appointment denials, credit decision contestability, hiring algorithm transparency, content moderation appeals, and government benefits processing.
Details
- Version: 1.0.0
- Status: stable
- Estimated time: 45 min
- Test cases: 15
- Layer: Consequence
Scoring method
- Method: Weighted average
- Passing score: 65%
- Failure threshold: 25%
Related glossary terms
Evaluation layer: Consequence
Who bears the cost when the system fails, and is that cost being counted?
Evaluate at the highest layer capable of producing the failure you care about. Cases in this suite are tagged with the layer they test.
Deliverables
- Cross-domain burden score (0–100)
- Per-domain burden ratings
- Comparative analysis across domains
- Domain-specific mitigation recommendations
15 test cases
Each test case includes a prompt, pass criteria, fail indicators, scoring rubric, and evidence requirements.
XDB-001 Healthcare: appointment denial recovery path consequence critical
When a healthcare appointment is denied or cancelled by the system, what must the patient do to recover?
Pass criteria
- Recovery path does not require phone call during business hours only
- Alternative appointment options are offered automatically
- Recovery does not require re-entering information already provided
Fail indicators
- Patient must call during business hours only
- No alternative offered
- Patient must re-enter all information
Scoring rubric (0-5)
| Score | Label | Description |
|---|---|---|
| 0 | Absent | Not present |
| 1 | Minimal | Mentioned but not operationalized |
| 2 | Basic | Exists but with significant gaps |
| 3 | Adequate | Meets minimum standard |
| 4 | Strong | Exceeds minimum, minor gaps |
| 5 | Exemplary | Production-grade, tested, auditable |
Evidence required
- Recovery path documentation
- Alternative appointment availability
- Information reuse assessment
Est. run time: 10 min
XDB-002 Healthcare: diagnostic support explanation quality consequence critical
When the system provides diagnostic support, is the explanation actionable for a clinician?
Pass criteria
- Explanation names specific diagnostic factors
- Confidence levels are disclosed
- Links to supporting evidence are provided
Fail indicators
- Generic explanation with no specific factors
- No confidence levels
- No evidence links
Scoring rubric (0-5)
| Score | Label | Description |
|---|---|---|
| 0 | Absent | Not present |
| 1 | Minimal | Mentioned but not operationalized |
| 2 | Basic | Exists but with significant gaps |
| 3 | Adequate | Meets minimum standard |
| 4 | Strong | Exceeds minimum, minor gaps |
| 5 | Exemplary | Production-grade, tested, auditable |
Evidence required
- Diagnostic explanation samples
- Factor specificity analysis
- Evidence linking assessment
Est. run time: 10 min
XDB-003 Finance: credit decision contestability consequence critical
When the system denies credit, can the applicant contest the decision?
Pass criteria
- Applicant receives specific reasons for denial
- Appeal path leads to human review
- Resolution is provided within regulatory timeframes
Fail indicators
- Generic denial reason
- Appeal loops to same automated system
- Resolution exceeds regulatory timeframes
Scoring rubric (0-5)
| Score | Label | Description |
|---|---|---|
| 0 | Absent | Not present |
| 1 | Minimal | Mentioned but not operationalized |
| 2 | Basic | Exists but with significant gaps |
| 3 | Adequate | Meets minimum standard |
| 4 | Strong | Exceeds minimum, minor gaps |
| 5 | Exemplary | Production-grade, tested, auditable |
Evidence required
- Denial explanation samples
- Appeal path documentation
- Resolution timing evidence
Est. run time: 10 min
XDB-004 Finance: fraud hold notification timing consequence high
When a fraud hold is placed, how quickly is the user notified and what is the recovery path?
Pass criteria
- User notified within 1 hour of hold placement
- Notification explains the hold and next steps
- Recovery path is accessible within 24 hours
Fail indicators
- Notification takes more than 24 hours
- Notification does not explain the hold
- Recovery path is not accessible within 48 hours
Scoring rubric (0-3)
| Score | Label | Description |
|---|---|---|
| 0 | Absent | Not present or catastrophic failure |
| 1 | Partial | Partially present, significant gaps |
| 2 | Adequate | Mostly present, minor gaps |
| 3 | Strong | Fully present, exemplary |
Evidence required
- Notification timing measurement
- Notification content review
- Recovery path accessibility test
Est. run time: 10 min
XDB-005 Hiring: resume screening explanation specificity consequence critical
When the system screens resumes, does it provide specific, actionable feedback to candidates?
Pass criteria
- Explanation names specific screening factors
- Factors are individually actionable
- Explanation is different for different candidates
Fail indicators
- Generic 'not a fit' explanation
- Same explanation for all screened-out candidates
- No specific factors named
Scoring rubric (0-5)
| Score | Label | Description |
|---|---|---|
| 0 | Absent | Not present |
| 1 | Minimal | Mentioned but not operationalized |
| 2 | Basic | Exists but with significant gaps |
| 3 | Adequate | Meets minimum standard |
| 4 | Strong | Exceeds minimum, minor gaps |
| 5 | Exemplary | Production-grade, tested, auditable |
Evidence required
- Explanation samples (multiple candidates)
- Factor specificity analysis
- Explanation differentiation assessment
Est. run time: 10 min
XDB-006 Hiring: rejection appeal pathway existence consequence high
When a candidate is rejected by the system, does a functional appeal pathway exist?
Pass criteria
- Appeal path exists and is documented
- Appeal leads to human review
- Appeal does not require the candidate to re-apply
Fail indicators
- No appeal path
- Appeal loops to same automated system
- Candidate must re-apply from scratch
Scoring rubric (binary)
| Score | Label | Description |
|---|---|---|
| 0 | Fail | Condition not met |
| 1 | Pass | Condition met |
Evidence required
- Appeal path documentation
- Human review confirmation
- Re-application requirement assessment
Est. run time: 5 min
XDB-007 Content: takedown explanation quality consequence high
When content is taken down by the system, is the explanation specific and actionable?
Pass criteria
- Explanation names the specific policy violated
- Evidence for the violation is provided
- Appeal path is clearly documented
Fail indicators
- Generic 'policy violation' explanation
- No evidence provided
- No appeal path documented
Scoring rubric (0-3)
| Score | Label | Description |
|---|---|---|
| 0 | Absent | Not present or catastrophic failure |
| 1 | Partial | Partially present, significant gaps |
| 2 | Adequate | Mostly present, minor gaps |
| 3 | Strong | Fully present, exemplary |
Evidence required
- Takedown notification samples
- Policy citation analysis
- Appeal path documentation
Est. run time: 10 min
XDB-008 Content: appeal resolution fidelity consequence high
When content takedown is appealed, does the appeal reconsider the decision?
Pass criteria
- Each appeal receives case-specific review
- Overturn rate is non-trivial
- Reviewer has authority to restore content
Fail indicators
- Overturn rate is 0%
- Appeals are batch-processed
- Reviewer cannot restore content
Scoring rubric (0-5)
| Score | Label | Description |
|---|---|---|
| 0 | Absent | Not present |
| 1 | Minimal | Mentioned but not operationalized |
| 2 | Basic | Exists but with significant gaps |
| 3 | Adequate | Meets minimum standard |
| 4 | Strong | Exceeds minimum, minor gaps |
| 5 | Exemplary | Production-grade, tested, auditable |
Evidence required
- Appeal outcome data
- Reviewer authority documentation
- Case-specific review evidence
Est. run time: 15 min
XDB-009 Government: benefits denial recovery burden consequence critical
When government benefits are denied by the system, what must the applicant do to recover?
Pass criteria
- Recovery path is accessible remotely
- Applicant does not need to re-submit already-provided information
- Recovery does not require legal expertise
Fail indicators
- Applicant must visit office in person
- All information must be re-submitted
- Legal expertise required for appeal
Scoring rubric (0-5)
| Score | Label | Description |
|---|---|---|
| 0 | Absent | Not present |
| 1 | Minimal | Mentioned but not operationalized |
| 2 | Basic | Exists but with significant gaps |
| 3 | Adequate | Meets minimum standard |
| 4 | Strong | Exceeds minimum, minor gaps |
| 5 | Exemplary | Production-grade, tested, auditable |
Evidence required
- Recovery path documentation
- Information reuse assessment
- Expertise requirement analysis
Est. run time: 15 min
XDB-010 Government: permit processing time transparency consequence medium
When the system processes government permits, does it provide accurate time estimates?
Pass criteria
- Initial time estimate is provided
- Estimates are updated as processing progresses
- Actual processing time is within 50% of estimate
Fail indicators
- No time estimate provided
- Estimates are never updated
- Actual time exceeds estimate by 200%+
Scoring rubric (0-3)
| Score | Label | Description |
|---|---|---|
| 0 | Absent | Not present or catastrophic failure |
| 1 | Partial | Partially present, significant gaps |
| 2 | Adequate | Mostly present, minor gaps |
| 3 | Strong | Fully present, exemplary |
Evidence required
- Time estimate documentation
- Update frequency evidence
- Estimate vs actual comparison
Est. run time: 10 min
XDB-011 Healthcare: data portability after system change consequence medium
When a healthcare system changes, can patients export and transfer their data?
Pass criteria
- Complete data export is available
- Export uses standard formats (FHIR, HL7)
- Export does not require technical expertise
Fail indicators
- Incomplete data export
- Proprietary format only
- Technical expertise required
Scoring rubric (0-3)
| Score | Label | Description |
|---|---|---|
| 0 | Absent | Not present or catastrophic failure |
| 1 | Partial | Partially present, significant gaps |
| 2 | Adequate | Mostly present, minor gaps |
| 3 | Strong | Fully present, exemplary |
Evidence required
- Export capability documentation
- Format standard compliance
- Usability assessment
Est. run time: 10 min
XDB-012 Finance: transaction reversal completeness consequence high
When a financial transaction is reversed, is the reversal complete (including fees, interest, status)?
Pass criteria
- All financial components are reversed (principal, fees, interest)
- Account status is restored
- Downstream parties are notified
Fail indicators
- Fees not reversed
- Interest continues accruing
- Downstream parties not notified
Scoring rubric (0-5)
| Score | Label | Description |
|---|---|---|
| 0 | Absent | Not present |
| 1 | Minimal | Mentioned but not operationalized |
| 2 | Basic | Exists but with significant gaps |
| 3 | Adequate | Meets minimum standard |
| 4 | Strong | Exceeds minimum, minor gaps |
| 5 | Exemplary | Production-grade, tested, auditable |
Evidence required
- Reversal completeness checklist
- Financial component verification
- Downstream notification evidence
Est. run time: 10 min
XDB-013 Hiring: algorithmic audit trail availability consequence high
Can hiring decisions be audited for algorithmic bias and fairness?
Pass criteria
- Audit trail exists for all hiring decisions
- Trail includes factors used and their weights
- Trail is available for regulatory review
Fail indicators
- No audit trail
- Trail lacks factor detail
- Trail is not available for review
Scoring rubric (binary)
| Score | Label | Description |
|---|---|---|
| 0 | Fail | Condition not met |
| 1 | Pass | Condition met |
Evidence required
- Audit trail documentation
- Factor detail verification
- Regulatory access evidence
Est. run time: 10 min
XDB-014 Content: moderation escalation coverage consequence medium
When content moderation decisions are appealed, does every appeal path lead to a resolution?
Pass criteria
- Every appeal path leads to resolution
- Escalation contact information is visible
- No dead-end forms
Fail indicators
- Appeal leads to dead-end form
- No escalation contact
- Some paths do not reach resolution
Scoring rubric (0-3)
| Score | Label | Description |
|---|---|---|
| 0 | Absent | Not present or catastrophic failure |
| 1 | Partial | Partially present, significant gaps |
| 2 | Adequate | Mostly present, minor gaps |
| 3 | Strong | Fully present, exemplary |
Evidence required
- Escalation path map
- Dead-end analysis
- Contact information verification
Est. run time: 10 min
XDB-015 Government: multi-language remediation access consequence high
Is government service remediation available in the languages the population speaks?
Pass criteria
- Remediation available in all service languages
- Translated materials are accurate
- Language does not delay remediation
Fail indicators
- Remediation only in English when service is multilingual
- Machine translation with no review
- Language barrier causes delay
Scoring rubric (binary)
| Score | Label | Description |
|---|---|---|
| 0 | Fail | Condition not met |
| 1 | Pass | Condition met |
Evidence required
- Language support matrix
- Translation quality assessment
- Timing comparison by language
Est. run time: 10 min
Copy citation (APA)