Worked case · Reference condition · 12 figures

Concurrent refund requests

Make retries financially idempotent

Figure 01 / 12

Dependencies form a critical path

Dependencies form a critical path — Concurrent refund requests. Illustrative service dependency graph. Exact values are in the figure data below.
Illustrative service dependency graph

The case examines make retries financially idempotent. Each edge is a required handoff in this illustrative path. A service can respond quickly while a downstream effect remains incomplete, so health needs both technical and business evidence.

Figure data and text version
FromToHandoff
Concurrent refund requestsDecision serviceAction request
Decision serviceEvidence providerRequired evidence
Decision serviceState storeDurable decision
State storeEffect publisherPending effect
Effect publisherExternal railFinancial action

Two workers receive a refund request while earlier attempts are still in flight. Correct arithmetic in each worker does not prevent a shared-state race.

The reference case starts with the stated population and a functioning evidence path. The owner is risk operations.

All amounts, rates, capacity limits, and outcomes in this case are synthetic. The three conditions are separate assumptions for comparison. A better result in the response condition is not measured proof that the proposed control causes that improvement. The figures expose the calculation and its limits; a real deployment needs its own evidence.

Read the result

85 intended requests generate 89 processing attempts under this retry assumption. Capacity is 120 attempts per interval, and the critical path consumes 150 ms of a 220 ms budget. The request-based SLO view observes 100 bad requests against an illustrative allowance of 100. These measurements must be connected to the financial effect and control evidence before declaring recovery.

Model inputs and calculated values

Inputs below are the case-specific values. Each figure states the condition-specific assumptions and units used in its calculation. Calculated values are rounded for display.

InputValue
arrivals85
capacity120
budgetMs220
Calculated valueResult
arrivals85
attempts89
capacity120
latency150
budget220
remaining70
window100,000
bad100
allowed100
Figure 02 / 12

Allocate the latency budget explicitly

Allocate the latency budget explicitly — Concurrent refund requests. Milliseconds on one critical path. Exact values are in the figure data below.
Milliseconds on one critical path

The four stages total 150 ms against a 220 ms budget, leaving 70 ms. Negative remaining time means this modeled path exceeds the target before adding any unmodeled overhead. The values are fixed teaching observations, not a latency guarantee.

Figure data and text version
StageMilliseconds
Ingress20
Feature reads35
Control evaluation70
Commit and response25
Remaining budget70
Figure 03 / 12

A slow minority changes the tail

A slow minority changes the tail — Concurrent refund requests. Milliseconds; explicitly constructed percentile profile. Exact values are in the figure data below.
Milliseconds; explicitly constructed percentile profile

The constructed distribution separates the median from high percentiles. Percentiles are order statistics over the same request population; summing stage p99 values is not generally the service p99. This chart is a teaching profile, not a measured production distribution. Horizontal positions are the labeled observations or scenarios; equal spacing does not imply equal numerical increments.

Figure data and text version
PercentileLatency ms
p50105
p75135
p90165
p95210
p99345
Figure 04 / 12

Retries increase attempted work

Retries increase attempted work — Concurrent refund requests. Attempts per interval. Exact values are in the figure data below.
Attempts per interval

85 original requests produce 89 attempts under the stated average retry multiplier. The additional 4 attempts consume capacity even when their financial effect must remain idempotent. This simple model omits recursive retry storms across multiple layers.

Figure data and text version
Attempt typeCount
Original requests85
Additional attempts4
Figure 05 / 12

Attempted demand versus capacity

Attempted demand versus capacity — Concurrent refund requests. Attempts; measures are not all additive. Exact values are in the figure data below.
Attempts; measures are not all additive

The interval has capacity for 120 attempts against 89 attempted requests. The difference is 0 unserved attempts in this simplified window. An unserved attempt is not necessarily an unexecuted business action: reconcile the stable action identifier before retrying a financial effect.

Figure data and text version
MeasureAttempts
Attempted demand89
Processing capacity120
Served this interval89
Unserved this interval0
Figure 06 / 12

Overload leaves a durable backlog

Overload leaves a durable backlog — Concurrent refund requests. Attempts waiting at interval end. Exact values are in the figure data below.
Attempts waiting at interval end

The six intervals use explicitly varied arrival multipliers and constant capacity. The queue carries forward unfinished attempts. Real systems also need a maximum age, admission policy, and expiry semantics so delayed work does not execute after its business authority has ended. Horizontal positions are the labeled observations or scenarios; equal spacing does not imply equal numerical increments.

Figure data and text version
IntervalPending attempts
T10
T20
T30
T432
T519
T60
Figure 07 / 12

Throughput can stay flat while demand rises

Throughput can stay flat while demand rises — Concurrent refund requests. Attempts per interval. Exact values are in the figure data below.
Attempts per interval

The completion count is limited by available work and capacity. A saturated completion line with a growing queue is evidence of overload, not stable end-to-end service. Inspect waiting time and customer outcomes alongside throughput. Horizontal positions are the labeled observations or scenarios; equal spacing does not imply equal numerical increments.

Figure data and text version
IntervalAttemptsCompletions
T16262
T28080
T3116116
T4152120
T5107120
T67190
Figure 08 / 12

Write state and pending effect together

Write state and pending effect together — Concurrent refund requests. Illustrative reliable handoff. Exact values are in the figure data below.
Illustrative reliable handoff

This sequence describes a transactional outbox within one datastore boundary. The publisher can deliver more than once, so the consumer also needs duplicate handling. A durable local handoff does not create a universal exactly-once guarantee across an external payment system.

Figure data and text version
EventCommit boundaryEvidence
Validate actionBefore transactionStable action key and permitted operation
Write state and outboxOne local transactionBoth records commit or neither does
Publish effectAfter commitDelivery may be retried
Consume effectConsumer boundaryIdempotent handling and stored result
ReconcileIndependent checkExpected and observed effects agree
Figure 09 / 12

Repeated delivery and financial effects differ

Repeated delivery and financial effects differ — Concurrent refund requests. Counts at different boundaries. Exact values are in the figure data below.
Counts at different boundaries

The example contains 89 transport attempts for 85 intended business actions. The protected effect count cannot be inferred from transport success alone. The table states the intended contract and the evidence needed to check it.

Figure data and text version
MeasureIllustrative valueRequired evidence
Business actions85Stable unique action identifiers
Transport attempts89Delivery identifiers and retries
Permitted effects85One effect per authorized action
Observed effectsMust reconcileAuthoritative ledger or external record
Figure 10 / 12

An SLO budget is a measured allowance

An SLO budget is a measured allowance — Concurrent refund requests. Request-based illustrative SLO; one measurement window. Exact values are in the figure data below.
Request-based illustrative SLO; one measurement window

For 100,000 eligible requests and a 99.9% illustrative SLO, the budget is 100 bad requests. This case observes 100. Eligibility, success, and measurement windows must be fixed before interpreting the result. An SLO allowance never overrides a legal or financial correctness requirement.

Figure data and text version
MeasureRequests
Eligible population100,000
Bad requests allowed100
Bad requests observed100
Remaining budget0
Figure 11 / 12

Dependency failures require scoped behavior

Dependency failures require scoped behavior — Concurrent refund requests. Dependency-specific policy boundary. Exact values are in the figure data below.
Dependency-specific policy boundary

The dependency is transactional datastore. The response depends on the operation, required control, and evidence available. The table gives illustrative behavior classes, not a universal fail-open rule.

Figure data and text version
Failure stateIllustrative responseEvidence to retain
Optional signal lateUse approved degraded policyMissing signal and selected policy version
Required control unavailableHold the affected actionPending owner and expiry
Unknown external outcomeQuery and reconcileOriginal external action identifier
Publisher retryRepeat delivery safelyStable business key and attempt history
Figure 12 / 12

Recovery ends with a reconciled population

Recovery ends with a reconciled population — Concurrent refund requests. Business recovery sequence. Exact values are in the figure data below.
Business recovery sequence

The last successful health probe does not close the incident. Recovery must classify affected actions, resolve unknown outcomes, restore required controls, and verify money and records. A replay is allowed only under the action’s current authority and idempotency contract.

Figure data and text version
StageCompletion evidence
ContainBoth requests consume the same remaining refundable amount.
ClassifyKnown success, known failure, unknown outcome
ResolveRepair or replay by stable action identifier
VerifyCounts, amounts, and control evidence reconcile
CloseOwners accept remaining exceptions explicitly

Connect the result to the system

Use one action key and an atomic state transition with defined conflict handling.

Check the population, evidence, permitted action, and actual effect together. A balanced calculation can still use the wrong population; a successful response can still leave an unknown financial outcome. The case’s numerical result applies only to its stated assumptions.

Sources and further reading

The chapter sources support the concepts and scope. They do not prescribe the synthetic model rates.

  1. Stripe: idempotent requests (provider example)
  2. Stripe: PaymentIntent lifecycle (provider example)