Require structured answer output with verifiable citations
The model double returns a plausible answer with a nonexistent citation.
- Focused work estimate
- 3h 30m + prerequisites
- Priority in the scenario
- High
- Engineering practice
- Schema validation · Grounding
Estimated field mix
- Applied AI70%
- Security30%
Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.
Review it, then add it to your workspace.
The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.
Project context
A fictional internal support team needs answers from product guides. Some guides are outdated or restricted, and fluent unsupported answers would create operational mistakes.
Setup prerequisites
- Author a synthetic document collection with two tenants, conflicting versions, and a deterministic model double; no model account required.
Preceding work
Complete these dependencies, or supply their agreed outputs before taking this ticket.
Acceptance criteria
- Validate the output schema and byte limit.
- Resolve every citation against authorized retrieved source versions.
- Reject unsupported citation identities rather than repairing them silently.
Implementation constraints
- Access model behavior only through AiEvaluationProvider.
Verification to include
- Accept a supported answer with valid ranges.
- Reject malformed output, nonexistent sources, and cross-tenant citations.
Deliverables
- Answer validator.
Rollout and recovery
Keep invalid responses in a safe unavailable state; retain only sanitized failure metadata.
Value of the work
For the engineer: Practice retrieval boundaries, citation validation, and deterministic evaluation.
For the team: Create a reviewable assistant prototype with clear refusal, cost, and freshness behavior.
Evidence boundaries
Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.
Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.