# noCV engineering task library Content version 5 Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs. Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence. ## BRAG — Grounded support-document assistant A fictional internal support team needs answers from product guides. Some guides are outdated or restricted, and fluent unsupported answers would create operational mistakes. **Field:** Applied AI. **Suggested stack:** TypeScript, PostgreSQL, AiEvaluationProvider. **Engineer value:** Practice retrieval boundaries, citation validation, and deterministic evaluation. **Company value:** Create a reviewable assistant prototype with clear refusal, cost, and freshness behavior. **Delivery agreement:** Deliver a local assistant using provider interfaces and scripted outputs; live model quality remains unmeasured. ### Setup prerequisites - Author a synthetic document collection with two tenants, conflicting versions, and a deterministic model double; no model account required. ### Control sources Version documents and enforce retrieval scope. #### BRAG-101 — Define versioned source records for the support collection **Task · Medium priority · Foundational** noCV practice brief v5 · BRAG-101 · Grounded support-document assistant Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Control sources. Depends on: No preceding ticket. Difficulty: Foundational. Estimated focused work: 60 minutes; setup and prerequisite tickets are additional. Estimated field mix: Data engineering 60% · Database engineering 40%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. Two guides share a title but describe different product releases. Acceptance criteria - Assign immutable source-version identities. - Record product version, scope, and supersession separately. - Preserve old content when a revision is added. Implementation constraints - Use synthetic document IDs, not fabricated Evidence IDs. Verification - Retrieve both historical versions explicitly. - Reject an attempt to overwrite a sealed source version. Deliverables - Source metadata schema. Rollout and recovery: Import a small synthetic collection first; append corrections instead of editing history. Project prerequisites: Author a synthetic document collection with two tenants, conflicting versions, and a deterministic model double; no model account required. Engineer value: Practice retrieval boundaries, citation validation, and deterministic evaluation. Company value: Create a reviewable assistant prototype with clear refusal, cost, and freshness behavior. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### BRAG-102 — Apply tenant and document grants before retrieval ranking **Task · High priority · Advanced** noCV practice brief v5 · BRAG-102 · Grounded support-document assistant Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Control sources. Depends on: BRAG-101. Difficulty: Advanced. Estimated focused work: 210 minutes; setup and prerequisite tickets are additional. Estimated field mix: Security 60% · Applied AI 40%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. A highly relevant private guide appears in another tenant's search results. Acceptance criteria - Scope candidate sources before scoring. - Recheck grants before returning retrieved text. - Ensure denied sources do not influence snippets or counts. Implementation constraints - Authorization belongs in the retrieval service boundary. Verification - Retrieve an authorized guide. - Query an identical restricted guide from another tenant and verify no disclosure. Deliverables - Scoped retriever. Rollout and recovery: Fail closed on unknown grants; disable assistant responses if retrieval authorization fails. Project prerequisites: Author a synthetic document collection with two tenants, conflicting versions, and a deterministic model double; no model account required. Engineer value: Practice retrieval boundaries, citation validation, and deterministic evaluation. Company value: Create a reviewable assistant prototype with clear refusal, cost, and freshness behavior. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### BRAG-103 — Preserve source offsets when splitting documentation **Task · Medium priority · Intermediate** noCV practice brief v5 · BRAG-103 · Grounded support-document assistant Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Control sources. Depends on: BRAG-101. Difficulty: Intermediate. Estimated focused work: 150 minutes; setup and prerequisite tickets are additional. Estimated field mix: Data engineering 60% · Applied AI 40%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. Answer citations point to chunks that cannot be located in the original guide. Acceptance criteria - Retain source version and character ranges for each chunk. - Keep headings with their relevant text within configured limits. - Reject chunks whose ranges exceed the source. Implementation constraints - Choose deterministic splitting; do not fabricate missing text. Verification - Reconstruct cited text from saved offsets. - Detect altered source content and invalid ranges. Deliverables - Chunking pipeline. Rollout and recovery: Rebuild indexes under a new version; retain the previous complete index for recovery. Project prerequisites: Author a synthetic document collection with two tenants, conflicting versions, and a deterministic model double; no model account required. Engineer value: Practice retrieval boundaries, citation validation, and deterministic evaluation. Company value: Create a reviewable assistant prototype with clear refusal, cost, and freshness behavior. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. ### Constrain answers Validate citations, contradictions, and provider failures. #### BRAG-104 — Require structured answer output with verifiable citations **Task · High priority · Advanced** noCV practice brief v5 · BRAG-104 · Grounded support-document assistant Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Constrain answers. Depends on: BRAG-102, BRAG-103. Difficulty: Advanced. Estimated focused work: 210 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 70% · Security 30%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. The model double returns a plausible answer with a nonexistent citation. Acceptance criteria - Validate the output schema and byte limit. - Resolve every citation against authorized retrieved source versions. - Reject unsupported citation identities rather than repairing them silently. Implementation constraints - Access model behavior only through AiEvaluationProvider. Verification - Accept a supported answer with valid ranges. - Reject malformed output, nonexistent sources, and cross-tenant citations. Deliverables - Answer validator. Rollout and recovery: Keep invalid responses in a safe unavailable state; retain only sanitized failure metadata. Project prerequisites: Author a synthetic document collection with two tenants, conflicting versions, and a deterministic model double; no model account required. Engineer value: Practice retrieval boundaries, citation validation, and deterministic evaluation. Company value: Create a reviewable assistant prototype with clear refusal, cost, and freshness behavior. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### BRAG-105 — Expose conflicting guide versions instead of choosing silently **Story · High priority · Advanced** noCV practice brief v5 · BRAG-105 · Grounded support-document assistant Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Constrain answers. Depends on: BRAG-104. Difficulty: Advanced. Estimated focused work: 180 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 100%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. Two authorized sources disagree about the retention setting and neither is marked superseded. Acceptance criteria - Represent the contradiction with both source references. - Avoid presenting either value as settled. - Ask for product-version context or return an explicit unresolved answer. Implementation constraints - Do not infer authority from retrieval score alone. Verification - Resolve a clearly superseded guide correctly. - Keep equally current contradictory guides visibly unresolved. Deliverables - Contradiction response behavior. Rollout and recovery: Enable with conflict fixtures first; route unresolved advice to a support review workflow. Project prerequisites: Author a synthetic document collection with two tenants, conflicting versions, and a deterministic model double; no model account required. Engineer value: Practice retrieval boundaries, citation validation, and deterministic evaluation. Company value: Create a reviewable assistant prototype with clear refusal, cost, and freshness behavior. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### BRAG-106 — Treat instructions inside retrieved guides as untrusted content **Bug · High priority · Advanced** noCV practice brief v5 · BRAG-106 · Grounded support-document assistant Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Constrain answers. Depends on: BRAG-104. Difficulty: Advanced. Estimated focused work: 180 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 50% · Security 50%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. A guide includes text telling the assistant to reveal other documents. Acceptance criteria - Keep retrieved text outside system instruction authority. - Disallow tool calls or new retrieval scopes from document text. - Reject answers citing content outside the authorized retrieval set. Implementation constraints - Use bounded prompt-injection fixtures without real secrets. Verification - Answer a benign guide question. - Inject scope-changing instructions and verify access remains unchanged. Deliverables - Adversarial retrieval checks. Rollout and recovery: Block deployment if the boundary fails; keep source ingestion separate from execution authority. Project prerequisites: Author a synthetic document collection with two tenants, conflicting versions, and a deterministic model double; no model account required. Engineer value: Practice retrieval boundaries, citation validation, and deterministic evaluation. Company value: Create a reviewable assistant prototype with clear refusal, cost, and freshness behavior. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### BRAG-107 — Bound assistant time and token reservations per request **Task · High priority · Intermediate** noCV practice brief v5 · BRAG-107 · Grounded support-document assistant Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Constrain answers. Depends on: BRAG-104. Difficulty: Intermediate. Estimated focused work: 150 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 70% · Backend 30%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. Repeated retries can exceed the budget for a single support question. Acceptance criteria - Reserve a configured maximum budget before provider calls. - Bound retries and total elapsed time. - Release unused reservation after terminal completion. Implementation constraints - Record provider, model, schema, timeout, cost, version, and trace metadata safely. Verification - Complete a request within the reservation. - Simulate timeout and ignored cancellation without unbounded retries. Deliverables - Request budget controller. Rollout and recovery: Default to deterministic provider mode; fail closed when reservation cannot be obtained. Project prerequisites: Author a synthetic document collection with two tenants, conflicting versions, and a deterministic model double; no model account required. Engineer value: Practice retrieval boundaries, citation validation, and deterministic evaluation. Company value: Create a reviewable assistant prototype with clear refusal, cost, and freshness behavior. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. ### Evaluate limits Measure scoped behavior without overstating model quality. #### BRAG-108 — Evaluate grounded answering separately from retrieval quality **Task · High priority · Expert** noCV practice brief v5 · BRAG-108 · Grounded support-document assistant Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Evaluate limits. Depends on: BRAG-105, BRAG-106, BRAG-107. Difficulty: Expert. Estimated focused work: 360 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 60% · Quality engineering 40%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. A single answer score hides whether failures came from missing documents or unsupported generation. Acceptance criteria - Create separate retrieval and citation-validity checks. - Include answerable, absent, conflicting, and denied questions. - Report deterministic-double results separately from unmeasured live-model behavior. Implementation constraints - Use public synthetic expectations; do not embed hidden evaluator answers in product APIs. Verification - Detect a missing retrieval result. - Detect a fluent answer unsupported by retrieved text. Deliverables - Evaluation report with failure categories. Rollout and recovery: Require both boundaries before expanding the corpus; retain an explicit unmeasured label for live quality. Project prerequisites: Author a synthetic document collection with two tenants, conflicting versions, and a deterministic model double; no model account required. Engineer value: Practice retrieval boundaries, citation validation, and deterministic evaluation. Company value: Create a reviewable assistant prototype with clear refusal, cost, and freshness behavior. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### BRAG-109 — Prevent stale indexes from serving revoked source access **Bug · High priority · Intermediate** noCV practice brief v5 · BRAG-109 · Grounded support-document assistant Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Evaluate limits. Depends on: BRAG-102, BRAG-108. Difficulty: Intermediate. Estimated focused work: 150 minutes; setup and prerequisite tickets are additional. Estimated field mix: Security 50% · Applied AI 30% · Distributed systems 20%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. A document grant is revoked after its chunks have been cached. Acceptance criteria - Recheck current access on cached retrieval results. - Evict or suppress revoked entries without serving stale text. - Keep cache keys scoped to tenant and index version. Implementation constraints - Cache entries cannot become access grants. Verification - Serve an unchanged authorized cache entry. - Revoke access and deny the next response even before cache expiry. Deliverables - Revocation regression. Rollout and recovery: Prioritize denial over cache availability; clear affected caches during recovery. Project prerequisites: Author a synthetic document collection with two tenants, conflicting versions, and a deterministic model double; no model account required. Engineer value: Practice retrieval boundaries, citation validation, and deterministic evaluation. Company value: Create a reviewable assistant prototype with clear refusal, cost, and freshness behavior. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### BRAG-110 — Document the assistant's supported questions and limits **Chore · Low priority · Foundational** noCV practice brief v5 · BRAG-110 · Grounded support-document assistant Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Evaluate limits. Depends on: BRAG-108, BRAG-109. Difficulty: Foundational. Estimated focused work: 60 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 100%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. A prototype can sound ready for unrestricted customer support despite its small synthetic corpus. Acceptance criteria - List supported corpus and product versions. - Explain unavailable and conflicting-source responses. - State that deterministic validation does not measure live-model answer quality. Implementation constraints - Avoid capability claims beyond observed local checks. Verification - Follow one supported question to its source. - Verify unsupported questions receive the documented response. Deliverables - Operator and user-facing capability note. Rollout and recovery: Ship the note with the prototype; revise it whenever corpus or provider behavior changes. Project prerequisites: Author a synthetic document collection with two tenants, conflicting versions, and a deterministic model double; no model account required. Engineer value: Practice retrieval boundaries, citation validation, and deterministic evaluation. Company value: Create a reviewable assistant prototype with clear refusal, cost, and freshness behavior. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence.