Open the refund circuit without declaring timed-out refunds failed
The synthetic refund provider begins timing out. The proposed circuit breaker maps every timeout to failed and releases queued retries when it closes, even though some timed-out refunds were accepted by the provider.
- Focused work estimate
- 3h 30m + prerequisites
- Priority in the scenario
- High
- Engineering practice
- Failure containment · Retry semantics · Provider reconciliation
Estimated field mix
- Distributed systems50%
- Site reliability30%
- Integrations20%
Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.
Pattern topics
- Circuit BreakerApply
Bound attempts during provider failure while keeping circuit health separate from each refund's durable and potentially uncertain business outcome.
Review it, then add it to your workspace.
The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.
Project context
A fictional equipment retailer lets customers choose a refund or replacement after inspection. Its payment and stock providers can time out after accepting an operation, so retrying the whole return is unsafe. Create a local TypeScript application, PostgreSQL state/outbox tables, and synthetic provider doubles with controllable outcomes; no starter code or fixtures are supplied. Use invented orders and integer minor-unit amounts only, with no real payments or external provider calls.
Setup prerequisites
- SQL transactions
- Idempotent commands
- Async failure handling
- State modeling
Preceding work
Complete these dependencies, or supply their agreed outputs before taking this ticket.
- PRECOVER-101 · Reject return decisions that skip inspection or reverse a completed refund
- PRECOVER-102 · Bind a repeated refund request to its original merchant and intent
- PRECOVER-103 · Commit approved return work and its dispatch record together
- PRECOVER-104 · Persist replacement progress across stock reservation and shipment creation
Acceptance criteria
- Define closed, open, and half-open behavior with a controllable clock, a bounded failure window, and a limited number of half-open probes.
- Opening the circuit defers new provider attempts without converting existing uncertain refund outcomes into confirmed failures.
- Reconcile ambiguous operation identities through the provider's status lookup before resubmission; a reopened circuit preserves pending work and retry timing.
Implementation constraints
- Use provider idempotency and status lookup contracts in addition to the breaker; a circuit breaker alone cannot prevent duplicate money movement.
Verification to include
- Advance a fake clock through the failure threshold, open window, and half-open success/failure paths and assert bounded probe calls.
- Simulate provider acceptance followed by timeout, then close the circuit; verify reconciliation finds the original refund and no second refund is created.
Deliverables
- Refund circuit policy, uncertain-outcome reconciliation, and clock-driven cases
Rollout and recovery
Observe the synthetic failure window before enabling deferral; disabling the breaker must not bypass uncertain-outcome reconciliation.
Value of the work
For the engineer: Practice durable workflow state, ambiguous provider outcomes, compensation, concurrency isolation, and recovery that survives process restarts.
For the team: Inspect whether a proposed workflow preserves refund and inventory invariants, contains provider failures, and gives operators a bounded recovery path.
Evidence boundaries
Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.
Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.