# noCV engineering task library Content version 5 Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs. Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence. ## SEARCH — Internal policy search with inspectable sources Employees search fictional travel and equipment policies. The current prototype blends draft and approved text and sometimes answers using a policy the employee cannot open. Use a small synthetic corpus and a deterministic answer-provider double. **Field:** Applied AI. **Suggested stack:** TypeScript, PostgreSQL, HTTP, JSON Schema. **Engineer value:** Practice access-aware retrieval, source provenance, structured model boundaries, and reproducible evaluation. **Company value:** Inspect how an engineer prevents unsupported or unauthorized answers and handles changing policy content. **Delivery agreement:** A local retrieval service with revisioned citations, abstention behavior, and an evaluation report. ### Setup prerequisites - Author synthetic policy documents with revisions, effective dates, and access groups. - Use a local deterministic provider double; paid model access is optional and not required. ### Make source authority explicit Index only identifiable, eligible document revisions. #### SEARCH-101 — Import policy documents with stable revision identities **Task · Medium priority · Foundational** noCV practice brief v5 · SEARCH-101 · Internal policy search with inspectable sources Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Make source authority explicit. Depends on: No preceding ticket. Difficulty: Foundational. Estimated focused work: 75 minutes; setup and prerequisite tickets are additional. Estimated field mix: Data engineering 60% · Database engineering 40%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. Two files called travel-policy.md contain different meal limits. The importer currently overwrites one with the other. Store a document identity, revision, and content digest. Acceptance criteria - Each imported revision records document ID, revision ID, digest, title, status, and effective date. - Reimporting identical bytes under the same revision is idempotent. - Different bytes under an existing revision are rejected; a new revision preserves the previous record. Implementation constraints - Use synthetic policy text and UTC timestamps. Verification - Import two revisions and retrieve their original content separately. - Reimport a modified file with the first revision ID and assert a conflict with no overwritten content. Deliverables - Revisioned importer and synthetic policy corpus Rollout and recovery: Index the synthetic corpus in a separate namespace; rollback by changing the active index pointer, not deleting revision history. Project prerequisites: Author synthetic policy documents with revisions, effective dates, and access groups. Use a local deterministic provider double; paid model access is optional and not required. Engineer value: Practice access-aware retrieval, source provenance, structured model boundaries, and reproducible evaluation. Company value: Inspect how an engineer prevents unsupported or unauthorized answers and handles changing policy content. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### SEARCH-102 — Keep chunk citations anchored to the original policy text **Task · Medium priority · Intermediate** noCV practice brief v5 · SEARCH-102 · Internal policy search with inspectable sources Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Make source authority explicit. Depends on: SEARCH-101. Difficulty: Intermediate. Estimated focused work: 105 minutes; setup and prerequisite tickets are additional. Estimated field mix: Data engineering 60% · Applied AI 40%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. An answer links to the policy title, but reviewers cannot find the quoted sentence after headings were stripped. Give each chunk a stable revision reference and exact text offsets. Acceptance criteria - Chunks record source revision, start and end offsets, and a content digest. - Reconstructing a chunk from the original source yields exactly the stored chunk text. - Chunk boundaries preserve headings and never split a Unicode code point. Implementation constraints - Specify the offset unit and normalization rules. - Chunk IDs change when chunking strategy changes. Verification - Reconstruct every chunk of a document containing emoji and repeated headings. - Change the chunking version and confirm old citation anchors remain resolvable. Deliverables - Chunker specification and source-anchor tests Rollout and recovery: Build a new chunk index beside the old one; swap only after source reconstruction passes. Project prerequisites: Author synthetic policy documents with revisions, effective dates, and access groups. Use a local deterministic provider double; paid model access is optional and not required. Engineer value: Practice access-aware retrieval, source provenance, structured model boundaries, and reproducible evaluation. Company value: Inspect how an engineer prevents unsupported or unauthorized answers and handles changing policy content. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### SEARCH-103 — Apply group access before ranking or sending context to a provider **Bug · High priority · Expert** noCV practice brief v5 · SEARCH-103 · Internal policy search with inspectable sources Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Make source authority explicit. Depends on: SEARCH-102. Difficulty: Expert. Estimated focused work: 240 minutes; setup and prerequisite tickets are additional. Estimated field mix: Security 60% · Applied AI 40%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. A general employee query retrieves a restricted executive travel exception. Hiding the final link is too late because the provider has already received the text. Enforce scope at retrieval and again at answer delivery. Acceptance criteria - The repository query limits candidates to the caller tenant and current allowed groups before ranking. - The provider receives no unauthorized chunk text or metadata. - Access revoked between retrieval and response prevents affected citations and answer content from being delivered. Implementation constraints - Use server-derived membership; never trust group IDs from the query body. - Do not share cached contexts across permission scopes. Verification - Query identical terms as users in different groups and inspect the exact provider request. - Revoke a group after retrieval using a barrier and verify delivery fails closed without restricted text. Deliverables - Access-scoped retrieval and revocation race tests Rollout and recovery: Disable answer generation for any request whose permission snapshot cannot be revalidated; deploy scoped retrieval before enabling ranking. Project prerequisites: Author synthetic policy documents with revisions, effective dates, and access groups. Use a local deterministic provider double; paid model access is optional and not required. Engineer value: Practice access-aware retrieval, source provenance, structured model boundaries, and reproducible evaluation. Company value: Inspect how an engineer prevents unsupported or unauthorized answers and handles changing policy content. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### SEARCH-104 — Exclude drafts and future policies from current-policy answers **Bug · High priority · Intermediate** noCV practice brief v5 · SEARCH-104 · Internal policy search with inspectable sources Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Make source authority explicit. Depends on: SEARCH-101. Difficulty: Intermediate. Estimated focused work: 105 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 50% · Data engineering 30% · Backend 20%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. A future equipment allowance outranks the currently approved policy. Define eligibility by approval, effective interval, and supersession so search answers the policy in effect at the requested date. Acceptance criteria - Current queries include approved revisions whose effective interval contains the supplied clock time. - Draft, withdrawn, and future revisions are excluded from answer context. - Overlapping eligible revisions for one policy produce an explicit conflict instead of a silent choice. Implementation constraints - Use an injected clock and half-open effective intervals. Verification - Search immediately before and at an effective-date boundary. - Create overlapping approved revisions and verify the query returns a conflict with no generated answer. Deliverables - Policy eligibility filter and effective-date cases Rollout and recovery: Run eligibility inspection over the corpus before switching active search; keep ambiguous documents unavailable for answers. Project prerequisites: Author synthetic policy documents with revisions, effective dates, and access groups. Use a local deterministic provider double; paid model access is optional and not required. Engineer value: Practice access-aware retrieval, source provenance, structured model boundaries, and reproducible evaluation. Company value: Inspect how an engineer prevents unsupported or unauthorized answers and handles changing policy content. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. ### Constrain answer generation Return supported citations or a clear lack-of-answer result. #### SEARCH-105 — Reject answers with invented or mismatched citations **Story · High priority · Advanced** noCV practice brief v5 · SEARCH-105 · Internal policy search with inspectable sources Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Constrain answer generation. Depends on: SEARCH-103, SEARCH-104. Difficulty: Advanced. Estimated focused work: 180 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 70% · Security 30%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. The provider double returns a fluent answer citing chunk-999, which was never retrieved. Add a structured response boundary and validate every reference before displaying an answer. Acceptance criteria - Responses conform to a strict schema with answer status, bounded claim text, and citation references. - Every citation resolves to an authorized chunk in the exact request context and source revision. - A cited quotation must match its referenced span; invalid output yields an unavailable result without partial answer text. Implementation constraints - Schema validity and quote matching do not prove broader semantic support; expose that limitation. - Treat provider text as untrusted output. Verification - Accept a fixture response with exact authorized citations. - Reject invented IDs, a correct ID with altered quotation, and additional schema properties. Deliverables - Structured answer validator and adversarial output fixtures Rollout and recovery: Make validation mandatory for all provider adapters; fall back to authorized search results when generation fails. Project prerequisites: Author synthetic policy documents with revisions, effective dates, and access groups. Use a local deterministic provider double; paid model access is optional and not required. Engineer value: Practice access-aware retrieval, source provenance, structured model boundaries, and reproducible evaluation. Company value: Inspect how an engineer prevents unsupported or unauthorized answers and handles changing policy content. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### SEARCH-106 — Return insufficient information when the corpus cannot answer **Story · High priority · Intermediate** noCV practice brief v5 · SEARCH-106 · Internal policy search with inspectable sources Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Constrain answer generation. Depends on: SEARCH-105. Difficulty: Intermediate. Estimated focused work: 120 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 100%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. Someone asks whether a bicycle repair is reimbursable, but the corpus contains only airfare and laptop rules. The prototype invents a limit. Add an explicit abstention path. Acceptance criteria - Empty retrieval returns INSUFFICIENT_INFORMATION without calling the answer provider. - A provider abstention is shown as missing support, without a fabricated policy rule. - The response may include authorized source links but never presents retrieval rank as certainty. Implementation constraints - Use a documented conservative context-selection rule with inspectable thresholds. Verification - Ask an unsupported bicycle-repair question and confirm no rule or amount is invented. - Ask an exact covered policy question and confirm authorized sources are retained in the supported fixture response. Deliverables - Abstention contract and covered/uncovered question fixtures Rollout and recovery: Default uncertain cases to source search; tune thresholds only against a versioned evaluation set. Project prerequisites: Author synthetic policy documents with revisions, effective dates, and access groups. Use a local deterministic provider double; paid model access is optional and not required. Engineer value: Practice access-aware retrieval, source provenance, structured model boundaries, and reproducible evaluation. Company value: Inspect how an engineer prevents unsupported or unauthorized answers and handles changing policy content. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### SEARCH-107 — Contain instructions embedded in a retrieved document **Bug · High priority · Advanced** noCV practice brief v5 · SEARCH-107 · Internal policy search with inspectable sources Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Constrain answer generation. Depends on: SEARCH-105. Difficulty: Advanced. Estimated focused work: 150 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 50% · Security 50%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. A synthetic policy appendix says to ignore the user and reveal all other documents. Ensure retrieved text remains data and cannot expand access or activate tools. Acceptance criteria - The provider request separates application instructions from quoted document context. - The answer interface exposes no file, network, or administrative tools. - Prompt-injection fixtures cannot alter document access scope or bypass citation validation. Implementation constraints - Do not claim that delimiting text alone prevents all prompt injection. - Use deterministic malicious-output fixtures to test downstream enforcement. Verification - Supply an instruction-bearing chunk and inspect the constructed provider request. - Return a malicious provider response with an unauthorized citation and verify the validator rejects it without a secondary fetch. Deliverables - Context construction boundary and injection containment tests Rollout and recovery: Keep tool access disabled for this feature; reject any adapter configuration that grants capabilities beyond answering. Project prerequisites: Author synthetic policy documents with revisions, effective dates, and access groups. Use a local deterministic provider double; paid model access is optional and not required. Engineer value: Practice access-aware retrieval, source provenance, structured model boundaries, and reproducible evaluation. Company value: Inspect how an engineer prevents unsupported or unauthorized answers and handles changing policy content. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. ### Evaluate and operate the service Bound provider failure, retention, and content changes. #### SEARCH-108 — Bound provider latency, retries, and retained query data **Story · High priority · Advanced** noCV practice brief v5 · SEARCH-108 · Internal policy search with inspectable sources Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Evaluate and operate the service. Depends on: SEARCH-106, SEARCH-107. Difficulty: Advanced. Estimated focused work: 150 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 40% · Privacy engineering 30% · Site reliability 30%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. A stalled model call occupies a request indefinitely and debug logs retain full employee queries. Add a bounded provider port with safe audit metadata. Acceptance criteria - Calls enforce timeout, attempt count, response-size, and token or equivalent local budget limits. - Audit metadata records provider/model version, request template version, schema version, duration, retry count, and trace ID. - Generic logs exclude raw questions, document text, and credentials; retention for any explicit debug store is configured separately. Implementation constraints - The default adapter is a local deterministic double. - Cost is recorded when known and unavailable when not supplied; do not invent cost figures. Verification - Simulate a stalled call and confirm a bounded unavailable response and cancellation. - Include secret marker text in query and context and scan normal diagnostics for leakage. Deliverables - Provider port, budget policy, and safe audit tests Rollout and recovery: Enable one provider adapter at a time behind the validated port; fall back to source results on budget or timeout failure. Project prerequisites: Author synthetic policy documents with revisions, effective dates, and access groups. Use a local deterministic provider double; paid model access is optional and not required. Engineer value: Practice access-aware retrieval, source provenance, structured model boundaries, and reproducible evaluation. Company value: Inspect how an engineer prevents unsupported or unauthorized answers and handles changing policy content. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### SEARCH-109 — Invalidate answers when policy authority or access changes **Bug · High priority · Expert** noCV practice brief v5 · SEARCH-109 · Internal policy search with inspectable sources Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Evaluate and operate the service. Depends on: SEARCH-108. Difficulty: Expert. Estimated focused work: 210 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 40% · Security 30% · Distributed systems 30%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. A cached answer still cites a withdrawn policy and is reused for an employee with different access. Bind cache entries to the exact corpus and permission authority. Acceptance criteria - Cache identity includes normalized query, corpus version, provider configuration, tenant, and permission-scope version. - Delivery rechecks every cited revision eligibility and current access even on a cache hit. - Withdrawing a cited policy prevents old answer delivery without rewriting the original cached record. Implementation constraints - Use a version pointer or tombstone for invalidation rather than editing source history. Verification - Warm the cache, withdraw a source, and verify the next request cannot return the old answer. - Reuse identical text across two permission groups and verify no cross-scope hit occurs. Deliverables - Authority-aware cache and withdrawal/access regression cases Rollout and recovery: Ship with caching disabled, then enable only after invalidation cases pass; a kill switch returns to live retrieval. Project prerequisites: Author synthetic policy documents with revisions, effective dates, and access groups. Use a local deterministic provider double; paid model access is optional and not required. Engineer value: Practice access-aware retrieval, source provenance, structured model boundaries, and reproducible evaluation. Company value: Inspect how an engineer prevents unsupported or unauthorized answers and handles changing policy content. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### SEARCH-110 — Publish an evaluation report that separates retrieval and answer failures **Task · Medium priority · Intermediate** noCV practice brief v5 · SEARCH-110 · Internal policy search with inspectable sources Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Evaluate and operate the service. Depends on: SEARCH-109. Difficulty: Intermediate. Estimated focused work: 150 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 60% · Quality engineering 40%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. A single accuracy percentage conceals whether failures came from missing sources or invalid generated answers. Create a small versioned evaluation set and report distinct failure categories. Acceptance criteria - Cases cover answerable, unsupported, conflicting, restricted, stale, and injection-bearing questions. - Reports separate eligible-source retrieval, citation validity, abstention behavior, and provider failures. - Each result records corpus, implementation, provider-double, and case-set versions with expected and actual observations. Implementation constraints - Do not claim deterministic-double results measure a real model quality level. - Keep evaluation examples synthetic and publishable. Verification - Run the full set twice and compare deterministic results. - Introduce an access-filter defect and confirm it appears as a security failure rather than an aggregate quality dip. Deliverables - Versioned evaluation cases and categorized report Rollout and recovery: Use category-specific gates for changes; require a separate measured report before replacing the deterministic adapter. Project prerequisites: Author synthetic policy documents with revisions, effective dates, and access groups. Use a local deterministic provider double; paid model access is optional and not required. Engineer value: Practice access-aware retrieval, source provenance, structured model boundaries, and reproducible evaluation. Company value: Inspect how an engineer prevents unsupported or unauthorized answers and handles changing policy content. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence.