Separate release blockers from tests awaiting investigation
The team reruns the whole suite until it turns green. Define a gate that retains the first failure, identifies explicit quarantines, and does not hide payment or access regressions.
- Focused work estimate
- 2h 15m + prerequisites
- Priority in the scenario
- Medium
- Engineering practice
- Quality strategy · CI policy · Failure analysis
Estimated field mix
- Quality engineering80%
- Platform engineering20%
Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.
Review it, then add it to your workspace.
The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.
Project context
A small commerce team has happy-path browser tests but still ships rounding and retry defects. Model a fictional shop using synthetic products, a local payment double, and explicit cart rules before adding regression coverage.
Setup prerequisites
- Create or provide a local checkout fixture application.
- Model products, tax rules, inventory, and a payment-provider double using synthetic data.
Preceding work
Complete these dependencies, or supply their agreed outputs before taking this ticket.
- CHECK-101 · Write the smallest cart fixture that catches a pricing regression
- CHECK-103 · Cover discount and tax rounding at the documented boundary
- CHECK-102 · Replace timing sleeps in the checkout happy path
- CHECK-107 · Verify checkout errors can be corrected using a keyboard
- CHECK-104 · Prove a lost payment response cannot create two orders
- CHECK-105 · Test the last-item race without relying on lucky timing
- CHECK-106 · Check that one customer cannot open another receipt
- CHECK-108 · Give each parallel test its own stock and customer records
- CHECK-109 · Capture enough failure detail without storing checkout secrets
Acceptance criteria
- Payment, inventory, pricing, and authorization regressions remain blocking even after a successful retry.
- Quarantined cases require an owner, issue reference, reason, and expiry date.
- Reports show first-attempt outcomes, retry outcomes, and omitted coverage separately.
Implementation constraints
- Use retries for diagnosis rather than rewriting the original result.
Verification to include
- Run a fail-then-pass fixture and verify its first failure remains visible and blocking when critical.
- Use an expired quarantine and confirm gate evaluation fails with the owning issue reference.
Deliverables
- Release-gate evaluator and quarantine policy examples
Rollout and recovery
Evaluate the new policy against recorded fixture runs before enabling enforcement; rollback policy configuration without deleting result history.
Value of the work
For the engineer: Practice risk-based test selection, stable browser automation, and diagnosis of intermittent failures.
For the team: Review whether a test suite catches business regressions and explains failures without creating noisy release gates.
Evidence boundaries
Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.
Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.