Reject answers with invented or mismatched citations
The provider double returns a fluent answer citing chunk-999, which was never retrieved. Add a structured response boundary and validate every reference before displaying an answer.
- Focused work estimate
- 3h + prerequisites
- Priority in the scenario
- High
- Engineering practice
- Structured output · Citation validation · Trust boundaries
Estimated field mix
- Applied AI70%
- Security30%
Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.
Review it, then add it to your workspace.
The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.
Project context
Employees search fictional travel and equipment policies. The current prototype blends draft and approved text and sometimes answers using a policy the employee cannot open. Use a small synthetic corpus and a deterministic answer-provider double.
Setup prerequisites
- Author synthetic policy documents with revisions, effective dates, and access groups.
- Use a local deterministic provider double; paid model access is optional and not required.
Preceding work
Complete these dependencies, or supply their agreed outputs before taking this ticket.
Acceptance criteria
- Responses conform to a strict schema with answer status, bounded claim text, and citation references.
- Every citation resolves to an authorized chunk in the exact request context and source revision.
- A cited quotation must match its referenced span; invalid output yields an unavailable result without partial answer text.
Implementation constraints
- Schema validity and quote matching do not prove broader semantic support; expose that limitation.
- Treat provider text as untrusted output.
Verification to include
- Accept a fixture response with exact authorized citations.
- Reject invented IDs, a correct ID with altered quotation, and additional schema properties.
Deliverables
- Structured answer validator and adversarial output fixtures
Rollout and recovery
Make validation mandatory for all provider adapters; fall back to authorized search results when generation fails.
Value of the work
For the engineer: Practice access-aware retrieval, source provenance, structured model boundaries, and reproducible evaluation.
For the team: Inspect how an engineer prevents unsupported or unauthorized answers and handles changing policy content.
Evidence boundaries
Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.
Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.