noCV
PPROVIDER-109 · Contain provider failure

Keep one slow provider from occupying every notification slot

Practice briefStoryAdvanced

Twenty stalled B requests consume the module's shared work pool. A's immediate responses cannot start, and callers keep adding pending requests until memory grows.

Focused work estimate
3h 30m + prerequisites
Priority in the scenario
High
Engineering practice
Concurrency limits · Backpressure · Resource lifecycle

Estimated field mix

  • Performance engineering40%
  • Site reliability40%
  • Integrations20%

Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.

Pattern topics

  • BulkheadApply

    Partition active and queued capacity by provider so stalled requests cannot exhaust another provider's execution slots or grow an unbounded queue.

Your next step

Review it, then add it to your workspace.

The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.

Project context

A fictional maintenance scheduler sends appointment notices through two providers. Their request shapes, error codes, and acknowledgement semantics differ. Build two local scripted provider doubles and a TypeScript application module; use synthetic recipients, block external network access, and never send real notifications. No provider accounts, starter repository, or production delivery qualification is supplied.

Setup prerequisites

  • HTTP contracts
  • Asynchronous cancellation
  • Dependency injection

Preceding work

Complete these dependencies, or supply their agreed outputs before taking this ticket.

Acceptance criteria

  • Give each provider at most three active attempts and five queued operations, with a typed overload result when its queue is full.
  • Queued cancellations remove work before execution; every completion or failure releases its slot exactly once.
  • A stalled B provider cannot consume A's admission capacity, and queue wait contributes to the caller's overall deadline.

Implementation constraints

  • Use bounded in-process queues and local promise gates; do not add an external broker or unbounded task buffers.

Verification to include

  • Saturate B, submit A work, and verify A starts promptly while B's active and queued counts stay within bounds.
  • Cancel queued work, throw inside an attempt, and resolve an attempt twice through a faulty stub; verify no slot leak or over-release.

Deliverables

  • Per-provider Bulkhead and overload/cancellation regression harness

Rollout and recovery

Start with the declared local limits and record saturation behavior; reject new work explicitly while draining queues before a configuration change.

Value of the work

For the engineer: Practice placing provider boundaries, comparing wrappers and coordinators, and testing ordering, cancellation, and resource limits under controlled faults.

For the team: Review whether a provider change can be made without rewriting scheduling rules, leaking recipient data, or turning one provider failure into wider exhaustion.

Evidence boundaries

Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.

Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.