Rehearse an aborted release and write the operator handoff
The release checklist says rollback tested, but nobody has timed recovery or tried it with an expanded database schema and pending worker jobs.
- Focused work estimate
- 3h + prerequisites
- Priority in the scenario
- Medium
- Engineering practice
- Incident drills · Runbooks · Recovery measurement
Estimated field mix
- Platform engineering50%
- Site reliability50%
Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.
Review it, then add it to your workspace.
The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.
Project context
A fictional scheduling service has an API, a worker, and PostgreSQL. Releases use immutable images on two application instances. The team needs compatibility checks, staged traffic, and a rehearsed rollback without introducing a new orchestration platform.
Setup prerequisites
- HTTP health checks
- CI pipelines
- Database migrations
Preceding work
Complete these dependencies, or supply their agreed outputs before taking this ticket.
- ROLL-102 · Separate liveness from database readiness
- ROLL-101 · Display the exact release identity in diagnostics
- ROLL-103 · Refuse deploys with incomplete runtime configuration
- ROLL-104 · Keep old API instances working during a column rename
- ROLL-105 · Prevent two release jobs from overwriting the same environment
- ROLL-106 · Base canary decisions on enough comparable requests
- ROLL-107 · Drain long requests before retiring an API instance
- ROLL-108 · Do not roll back a worker into an unreadable job format
- ROLL-109 · Attach a release marker to bounded operational telemetry
Acceptance criteria
- Run one simulated failed canary with pending jobs and expanded schema.
- Record detection, abort, drain, and recovery timestamps with observed limitations.
- Provide exact preconditions and stop conditions for the documented rollback path.
Implementation constraints
- A simulated drill establishes fixture behavior only; label its measured timings accordingly.
Verification to include
- Recover the previous compatible release and reconcile accepted jobs.
- Include an incompatible rollback example and prove the checklist tells the operator to stop.
Deliverables
- Recorded release drill and operator runbook
Rollout and recovery
Version the runbook with the tested manifests; repeat the bounded drill when compatibility assumptions change.
Value of the work
For the engineer: Practice release contracts, compatibility windows, measurable canaries, and incident decisions on a modest application topology.
For the team: Review whether an engineer can make deployments diagnosable and reversible while identifying when rollback is unsafe.
Evidence boundaries
Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.
Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.