# noCV engineering task library Content version 5 Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs. Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence. ## TRIAGE — Support routing that agents can correct A fictional software vendor receives billing, account-access, bug, and security reports. A prototype silently moves tickets based on vague model confidence. Replace it with bounded suggestions, deterministic safety rules, and an auditable review flow. **Field:** Applied AI. **Suggested stack:** TypeScript, PostgreSQL, JSON Schema, HTTP. **Engineer value:** Practice constrained classification, human correction workflows, model versioning, and evaluation under ambiguous inputs. **Company value:** Review whether automation saves triage effort while preserving queue ownership and safe escalation. **Delivery agreement:** A local routing service with reviewable suggestions, correction history, and a replay evaluation report. ### Setup prerequisites - Create a synthetic support-ticket corpus with no real customer messages. - Implement a deterministic local classifier double with success, malformed-output, and timeout modes. ### Define routing inputs and rules Make categories and required human handling explicit. #### TRIAGE-101 — Define queue labels with examples and an unknown outcome **Task · Medium priority · Foundational** noCV practice brief v5 · TRIAGE-101 · Support routing that agents can correct Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Define routing inputs and rules. Depends on: No preceding ticket. Difficulty: Foundational. Estimated focused work: 60 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 100%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. Different agents use Billing and Payments interchangeably, so routing reports cannot be compared. Define stable queue IDs with concise boundaries and ambiguous examples. Acceptance criteria - Every queue has a stable ID, description, included examples, and excluded examples. - UNKNOWN and NEEDS_REVIEW are explicit outcomes rather than invented queue names. - Changing a label definition creates a new taxonomy version. Implementation constraints - Use fictional support topics and avoid demographic categories. Verification - Map a duplicate-charge example and an account-reset example to distinct documented queues. - Validate that a mixed billing/security example can remain NEEDS_REVIEW. Deliverables - Versioned queue taxonomy and labeled examples Rollout and recovery: Use the taxonomy in suggestion-only mode; retain previous versions for historical reports. Project prerequisites: Create a synthetic support-ticket corpus with no real customer messages. Implement a deterministic local classifier double with success, malformed-output, and timeout modes. Engineer value: Practice constrained classification, human correction workflows, model versioning, and evaluation under ambiguous inputs. Company value: Review whether automation saves triage effort while preserving queue ownership and safe escalation. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### TRIAGE-102 — Normalize inbound messages without discarding the original record **Task · Medium priority · Intermediate** noCV practice brief v5 · TRIAGE-102 · Support routing that agents can correct Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Define routing inputs and rules. Depends on: TRIAGE-101. Difficulty: Intermediate. Estimated focused work: 90 minutes; setup and prerequisite tickets are additional. Estimated field mix: Data engineering 70% · Applied AI 30%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. Quoted email chains dominate routing input and duplicate attachments inflate request size. Create a bounded classification view while retaining the synthetic original message separately. Acceptance criteria - The derived view records normalization version, source message revision, and truncation indicators. - Input size is bounded and attachments contribute only permitted metadata. - The original record is immutable and can be inspected by an authorized support reviewer. Implementation constraints - Normalization is deterministic; do not use a model to silently rewrite the complaint. Verification - Normalize a long quoted chain twice and compare identical derived views. - Provide oversized text and an attachment filename containing markup; verify bounded plain-text input. Deliverables - Normalization pipeline and input-limit fixtures Rollout and recovery: Compare derived views in local review before enabling suggestions; use the original source reference for disputed cases. Project prerequisites: Create a synthetic support-ticket corpus with no real customer messages. Implement a deterministic local classifier double with success, malformed-output, and timeout modes. Engineer value: Practice constrained classification, human correction workflows, model versioning, and evaluation under ambiguous inputs. Company value: Review whether automation saves triage effort while preserving queue ownership and safe escalation. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### TRIAGE-103 — Route declared security incidents to review before calling a classifier **Story · High priority · Advanced** noCV practice brief v5 · TRIAGE-103 · Support routing that agents can correct Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Define routing inputs and rules. Depends on: TRIAGE-102. Difficulty: Advanced. Estimated focused work: 135 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 50% · Security 30% · Backend 20%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. A user selects the security-incident contact form, but the model labels the message a normal login issue. Respect trusted intake metadata and require the security review queue. Acceptance criteria - A trusted security-form source always produces a mandatory security-review state. - The classifier cannot downgrade that state or trigger a customer-facing reply. - Untrusted message text claiming to be system routing metadata does not acquire privileged routing authority. Implementation constraints - Distinguish server-owned intake fields from user-provided body text. - Do not infer incident severity from writing style or personal attributes. Verification - Submit a short login message through the trusted security form and confirm mandatory review without classification. - Put a forged security-source object in ordinary message text and confirm it remains untrusted content. Deliverables - Deterministic escalation policy and metadata-spoofing tests Rollout and recovery: Enable mandatory routing before model suggestions; keep a manual queue available when source metadata is missing. Project prerequisites: Create a synthetic support-ticket corpus with no real customer messages. Implement a deterministic local classifier double with success, malformed-output, and timeout modes. Engineer value: Practice constrained classification, human correction workflows, model versioning, and evaluation under ambiguous inputs. Company value: Review whether automation saves triage effort while preserving queue ownership and safe escalation. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. ### Offer bounded suggestions Validate and present proposals without silently moving tickets. #### TRIAGE-104 — Accept only bounded suggestions from the classifier **Story · High priority · Advanced** noCV practice brief v5 · TRIAGE-104 · Support routing that agents can correct Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Offer bounded suggestions. Depends on: TRIAGE-103. Difficulty: Advanced. Estimated focused work: 150 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 80% · Security 20%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. The provider returns a queue that no longer exists and a reason containing a link to an unrelated customer record. Validate suggestions against the exact taxonomy and source message. Acceptance criteria - Output contains only permitted queue IDs, a review flag, and bounded supporting spans from the input. - Every supporting span matches the normalized message exactly. - Malformed, out-of-taxonomy, or unsupported outputs become NEEDS_REVIEW with no partial suggestion. Implementation constraints - Do not treat a model-supplied confidence number as a calibrated probability. - Use strict schemas and plain text rendering. Verification - Accept a valid suggestion supported by an exact message span. - Reject invented queues, fabricated spans, extra action fields, and markup payloads. Deliverables - Suggestion validator and malformed-provider fixtures Rollout and recovery: Require the validator on every adapter; fall back to the unassigned review queue on rejection. Project prerequisites: Create a synthetic support-ticket corpus with no real customer messages. Implement a deterministic local classifier double with success, malformed-output, and timeout modes. Engineer value: Practice constrained classification, human correction workflows, model versioning, and evaluation under ambiguous inputs. Company value: Review whether automation saves triage effort while preserving queue ownership and safe escalation. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### TRIAGE-105 — Show a suggested queue as an explicit agent action **Story · Medium priority · Foundational** noCV practice brief v5 · TRIAGE-105 · Support routing that agents can correct Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Offer bounded suggestions. Depends on: TRIAGE-104. Difficulty: Foundational. Estimated focused work: 90 minutes; setup and prerequisite tickets are additional. Estimated field mix: Frontend 60% · Applied AI 20% · Accessibility 20%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. Agents cannot tell whether a ticket was assigned by a person or by the prototype. Display the suggestion alongside Accept and Choose another queue actions. Acceptance criteria - A suggestion never changes the assigned queue by itself. - The view labels provider version, suggestion time, and supporting message text. - Keyboard users can accept, dismiss, or choose another permitted queue. Implementation constraints - Do not describe the suggestion as a verified conclusion. Verification - Load a suggestion and verify persisted assignment remains unchanged until acceptance. - Dismiss it by keyboard and confirm no queue transition or customer message is created. Deliverables - Accessible suggestion panel and interaction tests Rollout and recovery: Release to an opt-in local agent view; disable the panel without changing existing queue assignment behavior. Project prerequisites: Create a synthetic support-ticket corpus with no real customer messages. Implement a deterministic local classifier double with success, malformed-output, and timeout modes. Engineer value: Practice constrained classification, human correction workflows, model versioning, and evaluation under ambiguous inputs. Company value: Review whether automation saves triage effort while preserving queue ownership and safe escalation. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### TRIAGE-106 — Discard a suggestion when the ticket changed while classification ran **Bug · High priority · Expert** noCV practice brief v5 · TRIAGE-106 · Support routing that agents can correct Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Offer bounded suggestions. Depends on: TRIAGE-104, TRIAGE-105. Difficulty: Expert. Estimated focused work: 210 minutes; setup and prerequisite tickets are additional. Estimated field mix: Backend 40% · Database engineering 30% · Applied AI 30%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. A customer adds a security-relevant reply while an older billing suggestion is in flight. The old suggestion must not become actionable against the new message revision. Acceptance criteria - A suggestion binds ticket ID, message revision, taxonomy version, and provider configuration. - Persisting or accepting a stale suggestion fails with an explicit revision conflict. - Concurrent acceptance by two agents creates at most one assignment transition and preserves both attempted-action outcomes. Implementation constraints - Enforce revision checks in the service/repository transaction, not only the UI. - Keep assignment audit records append-only. Verification - Pause classification, append a reply, then release the provider and confirm the suggestion is stale. - Race two acceptance requests and assert one transition with an explicit conflict for the loser. Deliverables - Revision-safe suggestion lifecycle and race tests Rollout and recovery: Enable acceptance only after revision checks are active; stale suggestions are regenerated or sent to manual review. Project prerequisites: Create a synthetic support-ticket corpus with no real customer messages. Implement a deterministic local classifier double with success, malformed-output, and timeout modes. Engineer value: Practice constrained classification, human correction workflows, model versioning, and evaluation under ambiguous inputs. Company value: Review whether automation saves triage effort while preserving queue ownership and safe escalation. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. ### Operate and learn from corrections Audit changes and compare revisions without leaking customer content. #### TRIAGE-107 — Keep provider outages from blocking the support inbox **Story · High priority · Intermediate** noCV practice brief v5 · TRIAGE-107 · Support routing that agents can correct Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Operate and learn from corrections. Depends on: TRIAGE-104. Difficulty: Intermediate. Estimated focused work: 120 minutes; setup and prerequisite tickets are additional. Estimated field mix: Distributed systems 40% · Applied AI 30% · Backend 30%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. The classifier stalls and newly submitted tickets disappear until its call completes. Decouple intake from classification and make failure an observable review state. Acceptance criteria - Ticket intake commits before asynchronous classification starts. - Jobs use deterministic identities and bounded timeout/retry settings. - Exhausted jobs leave tickets visible in manual review with a safe failure category. Implementation constraints - Use a transactional outbox or equivalent local transactional queue boundary. - The default provider is deterministic and requires no credentials. Verification - Simulate provider timeout and verify immediate ticket visibility plus eventual review state. - Deliver the same job twice and confirm it does not create duplicate suggestions. Deliverables - Asynchronous suggestion job and outage scenarios Rollout and recovery: Enable queued suggestions behind a feature flag; disable workers while preserving manual inbox intake. Project prerequisites: Create a synthetic support-ticket corpus with no real customer messages. Implement a deterministic local classifier double with success, malformed-output, and timeout modes. Engineer value: Practice constrained classification, human correction workflows, model versioning, and evaluation under ambiguous inputs. Company value: Review whether automation saves triage effort while preserving queue ownership and safe escalation. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### TRIAGE-108 — Record corrections without turning agent clicks into automatic training data **Story · Medium priority · Advanced** noCV practice brief v5 · TRIAGE-108 · Support routing that agents can correct Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Operate and learn from corrections. Depends on: TRIAGE-106. Difficulty: Advanced. Estimated focused work: 150 minutes; setup and prerequisite tickets are additional. Estimated field mix: Privacy engineering 40% · Backend 30% · Applied AI 30%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. A corrected queue overwrites the original suggestion, making disagreement impossible to investigate. Preserve the proposal, decision, actor, and reason as separate records. Acceptance criteria - Each correction links the original suggestion, selected queue, actor, time, and optional bounded reason. - Corrections are append-only and readable only within the ticket tenant and permitted support role. - No correction triggers external training or sends message text to a provider automatically. Implementation constraints - Treat corrections as operational feedback, not unquestionable ground truth. Verification - Correct one suggestion twice and inspect the complete ordered history. - Attempt to read another tenant correction and confirm denial with no message text leakage. Deliverables - Correction audit model and scoped history endpoint Rollout and recovery: Start collecting local operational feedback with retention controls; any later training export requires a separate governed workflow. Project prerequisites: Create a synthetic support-ticket corpus with no real customer messages. Implement a deterministic local classifier double with success, malformed-output, and timeout modes. Engineer value: Practice constrained classification, human correction workflows, model versioning, and evaluation under ambiguous inputs. Company value: Review whether automation saves triage effort while preserving queue ownership and safe escalation. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### TRIAGE-109 — Compare two routing revisions on the same ambiguous-ticket set **Task · Medium priority · Advanced** noCV practice brief v5 · TRIAGE-109 · Support routing that agents can correct Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Operate and learn from corrections. Depends on: TRIAGE-107, TRIAGE-108. Difficulty: Advanced. Estimated focused work: 150 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 60% · Quality engineering 40%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. A new prompt moves more messages to Billing, but the team cannot tell whether it improved routing or merely reduced abstention. Compare both revisions on a fixed synthetic case set. Acceptance criteria - Cases include mixed issues, short messages, quoted text, injection attempts, and explicit security intake. - Reports show per-queue outcomes, review rate, rule violations, invalid outputs, and changed decisions. - A mandatory-review violation blocks adopting the new revision regardless of aggregate routing accuracy. Implementation constraints - Pin normalization, taxonomy, provider-double, and case-set versions. - Keep abstentions visible rather than counting them as successful classifications. Verification - Run both revisions on identical inputs and reproduce the decision diff. - Inject a security downgrade into one revision and confirm it fails the adoption gate. Deliverables - Routing comparison harness and adoption report Rollout and recovery: Keep the current revision active while evaluating the new one; switch by versioned configuration after category gates pass. Project prerequisites: Create a synthetic support-ticket corpus with no real customer messages. Implement a deterministic local classifier double with success, malformed-output, and timeout modes. Engineer value: Practice constrained classification, human correction workflows, model versioning, and evaluation under ambiguous inputs. Company value: Review whether automation saves triage effort while preserving queue ownership and safe escalation. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence. #### TRIAGE-110 — Explain routing delays using metadata instead of customer messages **Chore · Medium priority · Intermediate** noCV practice brief v5 · TRIAGE-110 · Support routing that agents can correct Fictional engineering practice briefs. Starter repositories, fixtures, automated grading, and verified ownership are not included. Phase: Operate and learn from corrections. Depends on: TRIAGE-109. Difficulty: Intermediate. Estimated focused work: 105 minutes; setup and prerequisite tickets are additional. Estimated field mix: Applied AI 40% · Site reliability 30% · Privacy engineering 30%. Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence. Support leads need to diagnose slow suggestions, while logs currently include full complaint text. Add safe operational measures and an immediate suggestion kill switch. Acceptance criteria - Metrics expose queue delay, provider duration, timeout count, validation failures, and manual-review count without message bodies. - Audit metadata includes trace ID and exact provider/template/schema versions. - Disabling suggestions stops new provider calls while tickets and existing audit history remain accessible. Implementation constraints - Represent unavailable provider cost as unavailable, not zero. - Keep privileged access to source messages outside generic analytics. Verification - Run a delayed provider fixture and trace intake-to-review timing from metadata. - Toggle the kill switch with queued work and confirm no new calls and no lost tickets. Deliverables - Safe operations dashboard data and disable/recovery runbook Rollout and recovery: Enable metrics before activating suggestions; use the kill switch for unexpected routing changes while retaining manual service. Project prerequisites: Create a synthetic support-ticket corpus with no real customer messages. Implement a deterministic local classifier double with success, malformed-output, and timeout modes. Engineer value: Practice constrained classification, human correction workflows, model versioning, and evaluation under ambiguous inputs. Company value: Review whether automation saves triage effort while preserving queue ownership and safe escalation. AI tools are welcome during implementation. Record assumptions, review the result, and verify its behavior. Planning status does not create Outcome Evidence or Ownership Evidence.