Alert when a merchant close is blocked by stale imports
Operations sees a healthy API while three accepted imports have made no progress for 45 minutes. Finance discovers the blockage at the end of the day.
- Focused work estimate
- 2h + prerequisites
- Priority in the scenario
- Medium
- Engineering practice
- Observability · Alert design · Incident operations
Estimated field mix
- Site reliability70%
- Backend30%
Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.
Review it, then add it to your workspace.
The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.
Project context
A fictional subscription service invoices in USD and EUR. Finance currently compares a payment-provider CSV with database exports; late refunds and retried imports make the monthly close unreliable. Work on synthetic ledger entries only.
Setup prerequisites
- REST APIs
- SQL transactions
- Integer money representation
Preceding work
Complete these dependencies, or supply their agreed outputs before taking this ticket.
Acceptance criteria
- Expose accepted-to-completed age and counts by bounded processing state.
- Alert after the oldest accepted batch exceeds 15 minutes for two observations.
- The runbook identifies the batch safely and distinguishes retryable from invalid input failures.
Implementation constraints
- Avoid merchant IDs, file names, and references as metric labels.
Verification to include
- Advance a controlled clock to trigger a stale-batch alert.
- Confirm a rejected file does not trigger queue-lag paging.
Deliverables
- Metrics, alert rule, and import recovery runbook
Rollout and recovery
Observe alerts for one synthetic close cycle before enabling paging; disable the rule if it pages on rejected files.
Value of the work
For the engineer: Practice monetary invariants, concurrent writes, reconciliation, and operational recovery across one service boundary.
For the team: Review how an engineer makes discrepancies explainable and recoverable before a financial workflow is trusted.
Evidence boundaries
Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.
Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.