Commit telemetry rows and the batch checkpoint atomically
A process dies after inserting rows but before marking the file complete; replay doubles reported usage.
- Focused work estimate
- 4h + prerequisites
- Priority in the scenario
- High
- Engineering practice
- Transactions · Idempotency
Estimated field mix
- Data engineering50%
- Database engineering50%
Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.
Review it, then add it to your workspace.
The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.
Project context
A fictional energy dashboard receives hourly device batches. Corrupt archives and late corrections leave operators unsure which readings reached reports.
Setup prerequisites
- Create synthetic device batches and local object-store fixtures.
- Understand checksums and bounded streaming.
Preceding work
Complete these dependencies, or supply their agreed outputs before taking this ticket.
Acceptance criteria
- Persist rows and committed checkpoint in one transaction.
- Use source reading identity to reject duplicate insertion.
- Retries return committed counts without adding readings.
Implementation constraints
- Bound transaction size; larger batches require explicit sub-batch identities.
Verification to include
- Kill the test process at modeled commit boundaries.
- Replay a completed sub-batch and compare row counts and totals.
Deliverables
- Atomic batch loader and crash probe
Rollout and recovery
Canary small batches; pause loading and inspect manifests if reconciliation diverges.
Value of the work
For the engineer: Practice batch integrity, replay and data-quality boundaries.
For the team: Review whether operational data can be traced, corrected and recovered.
Evidence boundaries
Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.
Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.