Keep chunk citations anchored to the original policy text
An answer links to the policy title, but reviewers cannot find the quoted sentence after headings were stripped. Give each chunk a stable revision reference and exact text offsets.
- Focused work estimate
- 1h 45m + prerequisites
- Priority in the scenario
- Medium
- Engineering practice
- Text processing · Provenance · Unicode
Estimated field mix
- Data engineering60%
- Applied AI40%
Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.
Review it, then add it to your workspace.
The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.
Project context
Employees search fictional travel and equipment policies. The current prototype blends draft and approved text and sometimes answers using a policy the employee cannot open. Use a small synthetic corpus and a deterministic answer-provider double.
Setup prerequisites
- Author synthetic policy documents with revisions, effective dates, and access groups.
- Use a local deterministic provider double; paid model access is optional and not required.
Preceding work
Complete these dependencies, or supply their agreed outputs before taking this ticket.
Acceptance criteria
- Chunks record source revision, start and end offsets, and a content digest.
- Reconstructing a chunk from the original source yields exactly the stored chunk text.
- Chunk boundaries preserve headings and never split a Unicode code point.
Implementation constraints
- Specify the offset unit and normalization rules.
- Chunk IDs change when chunking strategy changes.
Verification to include
- Reconstruct every chunk of a document containing emoji and repeated headings.
- Change the chunking version and confirm old citation anchors remain resolvable.
Deliverables
- Chunker specification and source-anchor tests
Rollout and recovery
Build a new chunk index beside the old one; swap only after source reconstruction passes.
Value of the work
For the engineer: Practice access-aware retrieval, source provenance, structured model boundaries, and reproducible evaluation.
For the team: Inspect how an engineer prevents unsupported or unauthorized answers and handles changing policy content.
Evidence boundaries
Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.
Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.