Task library PIPE Data engineering · Phased project Make parcel tracking survive messy carrier events A fictional delivery marketplace receives JSON batches from three carriers. One uses local timestamps, another retries whole batches, and a third corrects delivery scans. Customer support needs a stable timeline rather than the last payload received.
Practice brief · Version 5
Project scope 10 tickets / 3 phases
Total focused work estimate 28h 30m + setup
Suggested stack TypeScript · PostgreSQL · BullMQ · S3-compatible storage Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included.
Choose a bounded subset for an assessment or practice session. Prerequisites and remaining project work must be agreed separately.
Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.
Pattern topics identify design choices to practice. Read the ticket's acceptance criteria and justify the simplest suitable approach. Tags are not capability or ownership evidence; an untagged ticket has no curated pattern topic assigned.
Recommended next step Start with PIPE-101 Open the first ticket for its prerequisites, acceptance criteria, verification plan, and an editable task draft.
What the engineer takes away Practice ingestion contracts, deduplication, event-time reconciliation, and replayable data repair.
What the team gains Examine how an engineer preserves source lineage and handles bad partner data without losing valid business events.
Before you start JSON schema validation SQL queries Event-time concepts Delivery agreement Ten tickets in three phases; hand off synthetic carrier fixtures, a replay procedure, and a support-readable tracking projection.
AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior.
Delivery phases PHASE 1 Keep valid events and explain rejected input.
PHASE 2 Resolve time, order, and corrections explicitly.
PHASE 3 Repair data safely and detect silent ingestion gaps.
Receive and quarantine Keep valid events and explain rejected input.
PIPE-101 Entry ticket; project setup still required Estimated field mix
Data engineering 80% API design 20% PIPE-102 Depends on PIPE-101 Estimated field mix
Data engineering 80% Storage systems 20% PIPE-103 Depends on PIPE-102 Estimated field mix
Data engineering 60% Database engineering 40% Build the tracking timeline Resolve time, order, and corrections explicitly.
PIPE-104 Depends on PIPE-102 PIPE-105 Depends on PIPE-103, PIPE-104 Estimated field mix
Data engineering 80% Backend 20% PIPE-106 Depends on PIPE-105 Estimated field mix
Data engineering 70% Database engineering 30% Replay and monitor Repair data safely and detect silent ingestion gaps.
PIPE-107 Depends on PIPE-106 Estimated field mix
Data engineering 60% Distributed systems 40% PIPE-108 Depends on PIPE-103 Estimated field mix
Data engineering 50% Performance engineering 30% Distributed systems 20% PIPE-109 Depends on PIPE-102 Estimated field mix
Data engineering 50% Site reliability 50% PIPE-110 Depends on PIPE-107 Estimated field mix
Privacy engineering 50% Data engineering 30% Storage systems 20% Use this project CSV keeps grouping and dependency keys as descriptive fields. Import mapping depends on your tracker configuration.