Add an ingestion freshness report that distinguishes missing data from zero
A site with no recent readings is displayed as consuming zero energy, misleading dashboard consumers.
- Focused work estimate
- 1h 30m + prerequisites
- Priority in the scenario
- Medium
- Engineering practice
- Data semantics · Observability
Estimated field mix
- Data engineering60%
- Site reliability40%
Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.
Review it, then add it to your workspace.
The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.
Project context
A fictional energy dashboard receives hourly device batches. Corrupt archives and late corrections leave operators unsure which readings reached reports.
Setup prerequisites
- Create synthetic device batches and local object-store fixtures.
- Understand checksums and bounded streaming.
Preceding work
Complete these dependencies, or supply their agreed outputs before taking this ticket.
- AINGEST-101 · Record a manifest before decoding a telemetry batch
- AINGEST-102 · Cap decompressed telemetry bytes before archive expansion
- AINGEST-103 · Normalize sensor units through a versioned conversion table
- AINGEST-104 · Commit telemetry rows and the batch checkpoint atomically
- AINGEST-105 · Quarantine malformed readings with usable row coordinates
- AINGEST-106 · Apply corrected readings without overwriting source history
- AINGEST-107 · Use event-time watermarks without discarding late telemetry silently
- AINGEST-108 · Reconcile uploaded telemetry manifests against committed checkpoints
- AINGEST-109 · Verify a telemetry backfill against immutable aggregate snapshots
Acceptance criteria
- Show last event time and last committed arrival separately.
- Represent missing intervals as unknown rather than numeric zero.
- Define stale thresholds from a documented expected schedule.
Implementation constraints
- Do not claim zero consumption without a reading.
Verification to include
- Report a genuine zero reading as zero.
- Remove a scheduled batch and show unknown with stale status.
Deliverables
- Freshness projection and missing-data cases
Rollout and recovery
Introduce freshness beside existing totals; revert display wiring while preserving unknown semantics.
Value of the work
For the engineer: Practice batch integrity, replay and data-quality boundaries.
For the team: Review whether operational data can be traced, corrected and recovered.
Evidence boundaries
Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.
Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.