Apple Card credit limits
The case to read the others against. The model was examined and cleared; the failure was that the person on the receiving end had no reasons and no working route, and the people answering the phone had neither either. Four of the six safeguards held. The two that did not, evidence and standing, are the two the contestability standard (STD-02) exists for.
- System
- Automated credit-limit decisions on a consumer card issued by Goldman Sachs, with no route by which an applicant could learn why their limit differed from a spouse's, and a reconsideration path that was a barrier: a six-month wait on a limit increase, removed only after the November 2019 complaints.
- Where
- United States · New York Department of Financial Services
- When
- Nov 2019 to 2021
- Scale
- No unlawful discrimination found. The regulator's finding was that applicants, and the bank's own staff, could not explain individual outcomes, and that the reconsideration path that existed was a barrier.
- Time to halt
- About seventeen months to a policy change; the model was not withdrawn.
- Errors carried by
- Spouses whose finances were shared but whose credit histories were not, offered lower limits with reconsideration gated behind a six-month wait. The bank's customer service staff took the complaints with no explanation to give.
- Halted by
- Nobody. The issuer changed its policies after a regulator's investigation found the process, not the model, deficient.
- After the failure
- Changed the rules
The record
What happened
From the primary sources listed at the end. Where a finding is quoted, it is the inquiry's, not ours.
- Nov 2019 Applicants report credit limits many times higher than their spouses', despite shared finances.
- Nov 2019 Customer service cannot explain the outcomes. By the applicants' accounts, staff say the algorithm decided. A higher limit carries a six-month wait.
- Nov 2019 Within days, the issuer removes the six-month wait on limit increases.
- Mar 2021 The New York regulator reports. The model did not use sex or marital status, and no fair lending law was broken.
- Mar 2021 It also finds that neither applicants nor the bank's staff could get that explanation at the time.
- 2021 The issuer adds joint accounts for spouses. The model is not changed.
In November 2019, several applicants reported publicly that they had been offered credit limits many times higher than their spouses', despite shared finances and, in some cases, the spouse's better credit history. Customer service representatives could not explain the outcomes and, by the applicants' accounts, said the algorithm had decided. The New York Department of Financial Services opened an investigation.
The Department's report, published in March 2021, found that the underwriting model did not use sex or marital status and that the outcomes could be explained by differences in the applicants' individual credit files, including that a spouse who was an authorized user on the other's accounts had a thinner history. It found no violation of fair lending law. It also found that neither applicants nor the bank's staff had been able to obtain that explanation at the time, that requesting a higher limit had carried a six-month wait removed only after the November 2019 complaints, and that the bank's reliance on individual credit data disadvantaged spouses whose finances were shared but whose credit histories were not.
The issuer removed the six-month wait within days of the November 2019 complaints, and after the report added the ability for spouses to share an account and build credit jointly. The model was not changed.
Scoring
The six safeguards
One finding per safeguard, with the clauses it falls under and the law it applies.
-
held
Capability
What could the assembled system do?
The model set limits. It did not do anything it had not been authorized to do, and its inputs were declared to the regulator.
-
held
Which actions was it permitted to take, for whom, until when?
The issuer held the authority to set limits and did so within fair lending law. The regulator examined the authority and confirmed it.
-
drifted
Evidence
What evidence justified that authority?
The evidence for each decision existed, in the credit file, and was produced to the regulator. It was not produced to the applicant, and the bank's own front line could not retrieve it. Evidence that the operator holds but cannot surface at the moment of the decision is evidence in name.
-
held
Dependency
How hard had withdrawal or substitution become?
A credit limit is revisable and the issuer revised its policy. Nothing about the deployment made withdrawal or change costly.
-
drifted
Standing
Who could challenge a decision, and did the challenge have to be answered?
Reconsideration existed but was a barrier. A credit-limit increase carried a six-month wait, removed only after the November 2019 complaints, after which limits were reconsidered within days. Complaints could be logged, but nothing was answered against a stated standard: neither applicants nor the bank's front line could get reasons naming the inputs that moved the limit, and there was no route to question the model. The regulator's finding was about this, not the model.
-
held
Correction
Which interventions were still possible, and how fast?
The issuer changed its policies within months of the report. It could make the change without a court, and it did. It took a regulator to prompt it; that it happened at all is the difference between this case and the other four.
Remediation
Avoid this failure in your system
The diagnostic that tests for this defect, the standard that would have required a fix, and the essay that explains why it recurs.
1. Run the diagnostic
Asks, for one workflow, what evidence each decision rests on, who bears the errors, whether they can challenge a decision, and who must answer, by when.
2. Apply the standard
STD-02: Contestability and recourse
Requires reasons with every decision, in plain language and at the moment it is made, and a published route for challenging it that names who answers and by when.
3. Read why it recurs
The model of a person is not the person
Why does a more accurate model of a person not remove that person's right to challenge a decision about them?
After the failure
Did the institution change?
Whether the institution changed the rule, category, or workflow that produced the errors, or corrected the errors and left that process as it was.
Changed the rules
The institution revised its machinery. The issuer had already removed the six-month wait within days of the November 2019 complaints and begun reconsidering limits within days; after the regulator found the process deficient, it changed the product so that spouses could share an account and build credit jointly. The model was not changed, and the reconsideration that existed never answered against a stated standard — no route gave an applicant reasons naming the inputs that moved the limit. This is the only case in the casebook where the correction was exercised by the operator rather than imposed from outside, and that is what made the episode end in institutional change rather than in settled cases.
Read the theory: exception learning · Corrective learning eval suite
The record that was not there
Every case reduces to a record the standards require and the operator did not hold. This is that record.
A record of who can challenge a credit-limit decision: that an applicant may ask for reasons, that the reasons name the inputs that moved the limit, and that a reconsideration is answered within a deadline. STD-02 §8.1 and §8.5 describe a process the issuer's policies never stated at the time.
The primary record
Courts, statutory inquiries, regulators, and parliaments, in that order of weight. Operator and NGO documents appear only where they record what the primary record established. No press coverage.
- Regulator New York State Department of Financial Services, Report on Apple Card Investigation · Mar 23, 2021