Replay a return workflow from its durable history after an orchestrator crash
A worker restarts halfway through a return and must decide which steps can resume without repeating side effects.
- Focused work estimate
- 6h + prerequisites
- Priority in the scenario
- Medium
- Engineering practice
- Recovery · Fault injection
Estimated field mix
- Distributed systems70%
- Site reliability30%
Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.
Review it, then add it to your workspace.
The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.
Project context
A fictional equipment-rental service approves returns through several provider calls. Partial failures leave labels issued without inspections or refunds reported before provider confirmation.
Setup prerequisites
- Create synthetic return records and fake inspection, shipping and refund adapters.
- Use integer money values and no real payments or messages.
Preceding work
Complete these dependencies, or supply their agreed outputs before taking this ticket.
- ASAGA-101 · Model return workflow steps as durable facts with explicit pending states
- ASAGA-102 · Define stable provider command keys for each return step
- ASAGA-103 · Write the compensation table for return side effects
- ASAGA-104 · Persist return progress and next-step dispatch atomically
- ASAGA-105 · Reconcile shipping acceptance before issuing a replacement label
- ASAGA-106 · Run label compensation only when its original effect is confirmed
- ASAGA-107 · Authorize refund progression from verified return facts
Acceptance criteria
- Reconstruct the current state from durable facts and generations.
- Reconcile outstanding provider commands before redispatch.
- Preserve completed facts and compensation history.
Implementation constraints
- Inject crashes between every modeled commit and provider response boundary.
Verification to include
- Replay a completed and a partially compensated synthetic return.
- Crash after refund acceptance and verify recovery does not issue a second refund.
Deliverables
- Workflow recovery harness and interruption matrix
Rollout and recovery
Run before enabling automatic recovery; pause ambiguous workflows while continuing unaffected ones.
Value of the work
For the engineer: Practice durable workflow state, compensation and uncertain outcomes.
For the team: Review recoverable business operations without assuming distributed transactions.
Evidence boundaries
Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.
Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.