Risk data contracts and event time
Build features that mean the same thing in analysis and production.
Only use facts known at decision time
Enlarge to read every label and explore the connections
A fact can occur before a decision and arrive after it. Reproducible features use event time and the time the system learned the fact.
Preserve occurrence and receipt time separately.
Exclude observations received after the decision.
Retain feature definitions and source versions.
The model is excellent in the notebook. In production, a missing timestamp becomes midnight and a missing amount becomes zero. The model did not change. The meaning of its inputs did.
Define an event contract
- ProduceEmit a versioned event
- ValidateCheck types units and required meaning
- ConsumeUse the documented contract
An event contract gives fields stable meaning. Include event_id, entity references, event_type, occurred_at, received_at, schema_version, and explicit money units. A payment amount without a currency is incomplete. A timestamp without a documented time basis is difficult to compare.
Validate at ingestion and preserve rejected records in a controlled repair path. Do not silently coerce invalid amounts or unknown event types into safe defaults. Additive schema changes still need compatibility tests when downstream consumers assume a complete field set. The contract is shared by producers and consumers, not owned only by the data warehouse.
Inside the mechanism. A durable event contract includes event ID, business action ID, entity references, event type, schema version, event time, receipt time, amount, currency, and source. Distinguish a corrected event from a repeated delivery. Validate semantic rules such as currency and amount compatibility, not only JSON shape. Rejected records need an owned quarantine path and reconciliation so validation cannot silently remove transactions from the risk population.
A concrete example. A transaction event needs a stable identity, schema version, event time, and amount with a currency. A producer change can break consumers even when the message still parses. The case identifies 12,250 eligible records from a source population of 12,500. The required workflow completes for 11,882, but 178 completed records miss the illustrative internal target. Another 368 remain incomplete. Communication evidence covers 11,763 generated notices. Scope, completion, timeliness, and delivery are four separate properties of the customer outcome.
When the assumption fails. A new producer reuses an identifier and changes the meaning of a nullable field. Version the contract, validate semantics, and quarantine ambiguous records before they update state. The following worked sequence shows the reference condition, a stress condition, and a response condition with explicit synthetic data. These are comparative assumptions, not measured causal effects.
A transaction event needs a stable identity, schema version, event time, and amount with a currency. A producer change can break consumers even when the message still parses.
- Occurred time
- When the underlying event happened
- Received time
- When this system learned about it
Event specimen
Illustrative data; not a real customer record or a prescribed policy.
- amount_minor12500
125.00 USD in this example
- currencyUSD
Explicit unit
- schema_version3
Consumer contract version
Safe-looking defaults can distort risk
Reject or repair invalid semantics explicitly. Safe-looking defaults can distort risk.
- Failure mode 1avoid
- Store money without currency. Amounts cannot be interpreted reliably.
- Failure mode 2avoid
- Use one timestamp for every purpose. Observation and arrival differ.
- Failure mode 3avoid
- Assume schema-valid means meaning-valid. Units and definitions still need checking.
Make time-aware features reproducible
- WindowBound event time
- KnowledgeExclude records learned later
- ReplayRebuild the feature at the original cutoff
A point-in-time feature uses only information available at the decision. Filter both event time and knowledge time when delayed data matters. An event that happened yesterday but arrived tomorrow was not available to today’s model.
For a one-hour count, exclude the current event and define whether the lower boundary is inclusive. The following teaching query counts earlier attempts known by the decision cutoff. Production code also needs the actual schema, indexes, deduplication, and transaction policy.
SELECT count(*)
FROM payment_events
WHERE account_id = :account_id
AND occurred_at >= :decision_time - interval '1 hour'
AND occurred_at < :decision_time
AND received_at <= :decision_time;
Inside the mechanism. Point-in-time correctness uses both when a fact occurred and when it became available. For a decision at time t, the feature join must exclude observations received after t even if their event date is earlier. Preserve the transformation version and input snapshot. This prevents a late dispute or corrected customer profile from leaking into a historical model evaluation. Time zones and window endpoints need explicit conventions.
A concrete example. A feature must contain only information available at the decision time. A late dispute cannot become an earlier transaction feature merely because its transaction date is old. The rule flags 659 of 18,200 decision snapshots. Of those flags, 393 meet the synthetic target, giving 59.64% precision. It misses 98 target events. Under the stated cost assumptions, residual loss and operating friction total $24,546. The important result is the connection between the population, action, capacity, and outcome—not one isolated score.
When the assumption fails. A training join uses the latest customer state and leaks future outcomes. Join by both event time and availability time, then preserve the feature snapshot. The following worked sequence shows the reference condition, a stress condition, and a response condition with explicit synthetic data. These are comparative assumptions, not measured causal effects.
A feature must contain only information available at the decision time. A late dispute cannot become an earlier transaction feature merely because its transaction date is old.
- Historical event
- Happened before the decision
- Available evidence
- Was also known before the decision
Late-arrival example
Illustrative data; not a real customer record or a prescribed policy.
- Occurred09:00
Earlier real event
- Received11:00
Later system knowledge
- Decision10:00
Cannot use the event then
Late data can create hidden leakage
Filter by availability as well as event time. Late data can create hidden leakage.
- Failure mode 1avoid
- Use the final warehouse snapshot. It includes facts learned later.
- Failure mode 2avoid
- Include the current attempt in prior history. That changes the feature definition.
- Failure mode 3avoid
- Ignore window boundaries. Off-by-one events can change decisions.
Preserve missingness and quality
- DetectIdentify missing stale or invalid evidence
- ClassifyRecord the cause where known
- FallbackUse the approved response for that defect
Missing data can come from a new customer, an unsupported source, a timeout, or a broken pipeline. These causes have different meanings. Use explicit validity and freshness fields rather than treating every missing value as zero.
Track completeness by source, product, and relevant population. A global 99 percent completeness rate can hide a fully broken small segment. Define which defects prevent a decision, which permit a bounded fallback, and which require later repair. The data-quality decision should be visible in the risk result so operations can distinguish customer risk from system uncertainty.
Missing is a state with possible causes. A device signal may be absent because the customer uses an unsupported environment, a provider is down, consent is unavailable, or the event arrived through a different product path. Replacing every absence with zero makes these situations look like the same measured value. Preserve the missing state and, where reliable and appropriate, its reason.
The decision policy must define what to do with that state. A model can be trained to handle missing inputs, but a new production outage may create a missingness pattern it never encountered during training. Monitor availability by feature and traffic segment. An aggregate health check can look normal while one high-impact population receives incomplete evidence.
Inside the mechanism. Use a typed quality state such as present, absent, stale, unavailable, or conflicting alongside the value. A missing count should not become zero unless the source contract establishes that meaning. Report completeness against the eligible population and by important segment. A global average can hide a broken partner feed. The decision record should retain the quality state actually used.
A concrete example. A zero count, an absent record, a stale response, and a failed source are distinct observations. Their operational meaning depends on the feature contract. The daily source population is 14,200 items, but 284 are outside the completed monitoring run. The included population creates 250 hits and 205 unique cases. With 17 cases already open and capacity for 240, the queue closes at 0. Coverage, duplicate work, and staffing are separate causes; reducing one number does not prove that the overall control improved.
When the assumption fails. The feature store replaces failed reads with zero and hides a device-data outage. Retain explicit quality states and route incomplete evidence through an approved policy. The following worked sequence shows the reference condition, a stress condition, and a response condition with explicit synthetic data. These are comparative assumptions, not measured causal effects.
A zero count, an absent record, a stale response, and a failed source are distinct observations. Their operational meaning depends on the feature contract.
- Known zero
- A valid measured absence
- Unknown value
- Measurement is unavailable or invalid
Quality record
Illustrative data; not a real customer record or a prescribed policy.
- history_countnull
Unavailable value
- qualityprovider_timeout
Known cause
- actionbounded fallback
Policy-defined treatment
System uncertainty must remain visible
Keep quality status in the decision record. System uncertainty must remain visible.
- Failure mode 1avoid
- Replace all nulls with zero. Unknown becomes a false measured fact.
- Failure mode 2avoid
- Monitor only global averages. Small segments can fail completely.
- Failure mode 3avoid
- Let every consumer invent a fallback. Behavior becomes inconsistent.
Use lineage as an engineering tool
- SourceIdentify original evidence
- TransformVersion the feature computation
- DecisionLink the resulting value to its use
Lineage connects a feature to its source events, transformations, and versions. It supports debugging, model review, customer corrections, and incident analysis. A column name is not enough if its definition changed over time.
Store the feature definition version and the source snapshot or reproducible reference needed for the use. Apply privacy controls to the retained data. When a source defect is found, use lineage to identify affected decisions and models. Without it, teams often rerun everything or miss part of the impact because they cannot trace which records consumed the bad field.
Inside the mechanism. Lineage connects a source fact to transformations, features, models, rules, actions, and later outcomes. Store stable version references and make the path inspectable for a historical decision. A dashboard definition change can alter reported performance without changing customer behavior. Version metric logic as well as model logic so a comparison can be reproduced under the same definitions.
A concrete example. An explanation must connect a decision to the evidence, transformations, policy, and model used at that moment. The current production model cannot explain every historical action. The case identifies 4,032 eligible records from a source population of 4,200. The required workflow completes for 3,911, but 59 completed records miss the illustrative internal target. Another 121 remain incomplete. Communication evidence covers 3,872 generated notices. Scope, completion, timeliness, and delivery are four separate properties of the customer outcome.
When the assumption fails. A feature transformation changes without a version and makes past decisions irreproducible. Retain immutable references to source facts and executable transformation versions. The following worked sequence shows the reference condition, a stress condition, and a response condition with explicit synthetic data. These are comparative assumptions, not measured causal effects.
An explanation must connect a decision to the evidence, transformations, policy, and model used at that moment. The current production model cannot explain every historical action.
- Column label
- Human-readable field name
- Lineage
- Traceable path from source to decision
Feature lineage
Illustrative data; not a real customer record or a prescribed policy.
- Featurerefund_ratio
Output field
- Definitionv8
Window and denominator rules
- Source batchbatch-117
Impact tracing reference
The same name can hide changed meaning
Version definitions and preserve traceable sources. The same name can hide changed meaning.
- Failure mode 1avoid
- Rename columns as the only history. Past calculations remain unclear.
- Failure mode 2avoid
- Keep raw data everywhere for convenience. Lineage still needs controlled access.
- Failure mode 3avoid
- Ignore downstream models during a data incident. They may inherit the defect.
Reconcile the population
- EligibleDefine the source population
- ProcessedTrace each required stage
- ReconcileExplain every material difference
Risk systems need population checks: eligible source events, accepted ingestion, feature computation, decisions, and downstream cases. Compare counts and identifiers across these stages. A model can be accurate on the records it sees while missing a large part of the business.
Use control totals and exception reports with explicit exclusions. Investigate unexpected differences by event type and partner. Preserve duplicates separately from missing events. A total count match can still hide one extra and one missing record, so perform identifier-level checks for consequential paths.
Population reconciliation tests whether the pipeline sees the activity it claims to cover. Compare source transactions with accepted events, feature rows, decisions, and final outcomes using stable identifiers and documented exclusions. An event can be valid in isolation while an entire partition is absent. Count checks, amount checks within currency, and age checks help locate these gaps. Keep duplicates, late arrivals, rejected records, and genuinely out-of-scope activity distinct so the reconciliation can explain a difference rather than merely detect one.
Inside the mechanism. Population reconciliation should conserve identifiers and values across stages while explaining legitimate exclusions and aggregation. Compare received, accepted, rejected, processed, and pending records for the same window. Account for late arrivals explicitly. A checkpoint or throughput counter is not an independent source of truth. Use authoritative upstream and downstream records to identify missing, duplicated, and mismatched actions.
A concrete example. Transport delivery counts do not establish that all financial events reached the ledger. Count and value comparisons need the same identifiers, window, and currency. The batch begins with $780,000 of instructions and $748,800.00 of captured value. At the observation cutoff, $22,464.00 remains pending. After the stated refunds, fees, and restrictions, $604,281.60 is available for payout. The unresolved instruction count is 2; an unknown external result is handled separately from a known decline.
When the assumption fails. A consumer checkpoint advances past rejected events and the dashboard still shows healthy throughput. Reconcile authoritative event IDs and amounts through ingestion, decision, and posting. The following worked sequence shows the reference condition, a stress condition, and a response condition with explicit synthetic data. These are comparative assumptions, not measured causal effects.
Transport delivery counts do not establish that all financial events reached the ledger. Count and value comparisons need the same identifiers, window, and currency.
- Model accuracy
- Quality on scored records
- System coverage
- Whether required records were scored at all
Coverage reconciliation
Illustrative data; not a real customer record or a prescribed policy.
- Eligible10000
Source events
- Decided9800
Risk results
- Unexplained200
Coverage gap despite good model accuracy
Coverage is a separate property from accuracy
Reconcile identifiers as well as totals. Coverage is a separate property from accuracy.
- Failure mode 1avoid
- Measure only scored records. Missing events disappear from the metric.
- Failure mode 2avoid
- Assume equal counts mean equal populations. Offsetting errors can remain.
- Failure mode 3avoid
- Exclude failures without reporting them. The denominator becomes misleading.
Chapter connections
Continue with Decision engines, rules, and reliable execution to follow the next part of the system. Use the glossary for terminology and risk mathematics for formulas and worked calculations.