noCV
QUEUE-106 · Handle failures deliberately

Fence a late worker after its rendering lease expires

Practice briefBugExpert

Worker A pauses long enough to lose its lease. Worker B completes the same preview, then A resumes and overwrites B's artifact pointer with stale output.

Focused work estimate
5h + prerequisites
Priority in the scenario
High
Engineering practice
Fencing tokens · Worker leases · Concurrency · Artifact consistency

Estimated field mix

  • Distributed systems60%
  • Platform engineering40%

Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.

Your next step

Review it, then add it to your workspace.

The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.

Project context

A fictional internal document portal creates previews through a trusted mock renderer. Large batches crowd out small teams, failed documents retry forever, and operators lack a safe replay command. This exercise never executes uploaded code or real document macros.

Setup prerequisites

  • Queue semantics
  • Database transactions
  • Operational metrics

Preceding work

Complete these dependencies, or supply their agreed outputs before taking this ticket.

Acceptance criteria

  • Associate each execution attempt with a monotonic fencing identity.
  • Accept completion only from the current authorized lease holder.
  • Keep rejected late completions observable and prevent them from changing terminal job state.

Implementation constraints

  • Lease expiration alone does not stop a process; the persistence boundary must reject stale writes.

Verification to include

  • Expire A, complete through B, then submit A's completion and inspect unchanged output.
  • Replay B's completion and assert one terminal transition and artifact reference.

Deliverables

  • Fenced completion operation and paused-worker reproduction

Rollout and recovery

Introduce fencing before extending concurrency; rollback dispatch while retaining fencing checks on completions.

Value of the work

For the engineer: Practice asynchronous lifecycle control, fair scheduling, bounded retries, and artifact recovery with a deterministic provider.

For the team: See how an engineer accounts for accepted work and reduces operational toil without granting operators broad data access.

Evidence boundaries

Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.

Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.