noCV
BINCIDENT-106 · Contain safely

Tune retry delay for a throttled dependency

Practice briefTaskIntermediate

Immediate retries amplify the dependency's temporary throttling.

Focused work estimate
2h 30m + prerequisites
Priority in the scenario
High
Engineering practice
Backoff · Resilience

Estimated field mix

  • Site reliability60%
  • Integrations40%

Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.

Your next step

Review it, then add it to your workspace.

The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.

Project context

A fictional document service has a rising processing backlog. Operators cannot tell whether arrivals, worker failures, or a slow dependency is responsible.

Setup prerequisites

  • Create a bounded local queue simulation with synthetic jobs and controllable worker/dependency failures.

Preceding work

Complete these dependencies, or supply their agreed outputs before taking this ticket.

Acceptance criteria

  • Respect bounded retry-after guidance.
  • Apply jitter with reproducible test control.
  • Cap total attempts and elapsed time.

Implementation constraints

  • Never retry permanent validation or authorization failures.

Verification to include

  • Recover after a temporary throttle.
  • Exhaust a persistent outage without a retry storm.

Deliverables

  • Dependency retry policy.

Rollout and recovery

Trial on one job class; pause dispatch if throttling remains sustained.

Value of the work

For the engineer: Practice incident timelines, queue diagnosis, and measured recovery.

For the team: Produce a repeatable response that preserves work integrity and exposes customer delay.

Evidence boundaries

Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.

Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.