Worked case · Controlled response · 12 figures

Early-warning response

Use early warnings with a response

Figure 01 / 12

The evaluated population

The evaluated population — Early-warning response. Count; one closed observation cohort. Exact values are in the figure data below.
Count; one closed observation cohort

The cohort contains 21,000 active credit accounts, of which 1,365 have the defined synthetic outcome: Synthetic material credit deterioration. The outcome is known by construction here. In production, label uncertainty and selection must be recorded separately.

Figure data and text version
OutcomeCount
Synthetic material credit deterioration1,365
Other labeled outcomes19,635

An early-warning signal is useful only when it leads to a proportionate and timely action. Precision, missed cases, and the effect of the intervention are different measures.

Evaluate mature outcomes and connect warnings to a defined review and response process.

All amounts, rates, capacity limits, and outcomes in this case are synthetic. The three conditions are separate assumptions for comparison. A better result in the response condition is not measured proof that the proposed control causes that improvement. The figures expose the calculation and its limits; a real deployment needs its own evidence.

Read the result

The rule flags 1,410 of 21,000 active credit accounts. Of those flags, 1,174 meet the synthetic target, giving 83.26% precision. It misses 191 target events. Under the stated cost assumptions, residual loss and operating friction total $433,300. The important result is the connection between the population, action, capacity, and outcome—not one isolated score.

Model inputs and calculated values

Inputs below are the case-specific values. Each figure states the condition-specific assumptions and units used in its calculation. Calculated values are rounded for display.

InputValue
population21,000
prevalence0.065
severity2,100
reviewCost22
Calculated valueResult
population21,000
positive1,365
negative19,635
tp1,174
fp236
fn191
tn19,399
loss401,100
severity2,100
precision83.26
recall86.01
Figure 02 / 12

Four outcomes of the rule

Four outcomes of the rule — Early-warning response. Counts; rows are actual labels, columns are actions. Exact values are in the figure data below.
Counts; rows are actual labels, columns are actions

The rule flags 1,174 synthetic positives and 236 negatives. It misses 191 positives. A flagged item is a decision to intervene; it is not proof of fraud, a legal prohibition, or any other real-world conclusion.

Figure data and text version
Known outcomeFlaggedNot flagged
Synthetic material credit deterioration1,174191
Other outcome23619,399
Figure 03 / 12

Three rates with different denominators

Three rates with different denominators — Early-warning response. Rates within this cohort. Exact values are in the figure data below.
Rates within this cohort

Precision is 83.26%, recall is 86.01%, and the false-positive rate is 1.2%. Changing the denominator changes the meaning. This record keeps each numerator attached to the population from which it came.

Figure data and text version
MetricNumeratorDenominatorResult
Precision1,1741,41083.26%
Recall1,1741,36586.01%
False-positive rate23619,6351.2%
Figure 04 / 12

Precision changes with prevalence

Precision changes with prevalence — Early-warning response. Percent; fixed conditional detection rates. Exact values are in the figure data below.
Percent; fixed conditional detection rates

This sensitivity plot holds recall at 86% and false-positive rate at 1.2%, then changes prevalence. It is an algebraic comparison, not a forecast. Even unchanged detection quality can produce a very different review queue when the base rate changes. Horizontal positions are the labeled observations or scenarios; equal spacing does not imply equal numerical increments.

Figure data and text version
Assumed prevalencePrecision %
0.1%6.69
0.5%26.48
1%41.99
2%59.39
5%79.04
10%88.84
Figure 05 / 12

The threshold trade-off

The threshold trade-off — Early-warning response. Count in the same cohort. Exact values are in the figure data below.
Count in the same cohort

Six illustrative score bands use a stated pair of detection rates. Lower sensitivity can reduce false alarms but miss more target events. These points do not come from a trained model and do not establish the best operating threshold. Horizontal positions are the labeled observations or scenarios; equal spacing does not imply equal numerical increments.

Figure data and text version
Score bandTrue positivesFalse positives
Band 11,3382,945
Band 21,2831,571
Band 31,174687
Band 4983236
Band 568279
Band 634120
Figure 06 / 12

A transparent loss-and-friction calculation

A transparent loss-and-friction calculation — Early-warning response. Illustrative USD; expected cost under stated intervention assumptions. Exact values are in the figure data below.
Illustrative USD; expected cost under stated intervention assumptions

At $2100 severity per missed synthetic positive, residual loss is $401,100. Review costs $31,020; lost contribution on false alarms is $1,180. The calculation assumes intervention prevents all flagged-positive loss and each false alarm loses the stated contribution. Relax those assumptions before applying it to a real policy.

Figure data and text version
Cost componentUSD
Missed-positive loss401,100
Review cost31,020
False-alarm contribution1,180
Figure 07 / 12

Observed outcomes mature over time

Observed outcomes mature over time — Early-warning response. Count; final outcome fixed. Exact values are in the figure data below.
Count; final outcome fixed

The final synthetic positive count is 1,365. Earlier observations reveal only a stated fraction. Comparing a day-1 cohort with a day-30 cohort would confuse label age with control quality. This curve models observation delay only; it does not change the final outcome. Horizontal positions are the labeled observations or scenarios; equal spacing does not imply equal numerical increments.

Figure data and text version
Days after eventObserved positives
1246
3478
7819
141,119
301,365
Figure 08 / 12

Review demand and available capacity

Review demand and available capacity — Early-warning response. Items in the cohort window. Exact values are in the figure data below.
Items in the cohort window

The flag count is 1,410. The comparison capacity is an illustrative 840 reviews per cohort window. A mathematical rule can be coherent while its resulting workload exceeds the operating team’s capacity. Capacity is not permission to ignore an applicable mandatory control.

Figure data and text version
Queue measureItems
Flagged for review1,410
Available capacity840
Excess demand570
Figure 09 / 12

A feature is an observation with provenance

A feature is an observation with provenance — Early-warning response. Illustrative data contract. Exact values are in the figure data below.
Illustrative data contract

This evidence contract supports use early warnings with a response. A value needs its event time, arrival time, scope, and source. Keeping unavailable evidence distinct from a measured zero prevents an outage from becoming a falsely reassuring feature.

Figure data and text version
FieldExampleMeaning
entity_refEarly-warning responseSubject of this case
event_time2026-09-18T09:00:00ZWhen the event occurred
received_time2026-09-18T09:00:02ZWhen the system learned it
signal_statusrepairedEvidence quality, not an outcome
label_definitionSynthetic material credit deteriorationThe target used in these calculations
Figure 10 / 12

Missing evidence changes the observed population

Missing evidence changes the observed population — Early-warning response. Count; each row is the same cohort. Exact values are in the figure data below.
Count; each row is the same cohort

The cells show an explicitly constructed completeness profile for three signal groups. The stress condition removes more history and device evidence. Missingness does not prove the target outcome; it changes what the decision process knows.

Figure data and text version
Signal groupAvailableMissing
Identity evidence20,895105
Activity history20,685315
Context signal20,370630
Figure 11 / 12

Evidence, score, and action remain separate

Evidence, score, and action remain separate — Early-warning response. Decision lifecycle. Exact values are in the figure data below.
Decision lifecycle

The policy can use use early warnings with a response only within its approved scope. The action record must retain which evidence was available, which model or rule ran, and which action was actually applied. The final action can differ from the score recommendation when a separate constraint applies.

Figure data and text version
StageRecord
ObserveEarly-warning response: evidence as of the decision time
EvaluateRule flags 1,410 of 21,000 active credit accounts
ApplyRecord action, reason, owner, and expiry
ReconcileJoin the action to later outcomes without overwriting history
Figure 12 / 12

What the result cannot establish

What the result cannot establish — Early-warning response. Interpretation boundary. Exact values are in the figure data below.
Interpretation boundary

Observed classifications do not reveal every counterfactual. The synthetic labels make arithmetic possible, but production decline data is selected by prior policy. Keep measured outcomes, assumed prevention, and unknown alternatives separate when reporting impact.

Figure data and text version
ClaimEvidence in this caseLimit
Detected target1174 known synthetic positives flaggedProduction labels may be delayed or wrong
Prevented lossAssumed 2,465,400 USDRequires an intervention-effect assumption
Customer impact236 synthetic negatives flaggedNot every flag causes abandonment
Unobserved alternativeOutcome without the actionNeeds a valid evaluation design

Connect the result to the system

Evaluate mature outcomes and connect warnings to a defined review and response process.

Check the population, evidence, permitted action, and actual effect together. A balanced calculation can still use the wrong population; a successful response can still leave an unknown financial outcome. The case’s numerical result applies only to its stated assumptions.

Sources and further reading

The chapter sources support the concepts and scope. They do not prescribe the synthetic model rates.

  1. OCC Comptroller’s Handbook: rating credit risk
  2. Regulation B, 12 CFR 1002.9: notifications