Record corrections without turning agent clicks into automatic training data
A corrected queue overwrites the original suggestion, making disagreement impossible to investigate. Preserve the proposal, decision, actor, and reason as separate records.
- Focused work estimate
- 2h 30m + prerequisites
- Priority in the scenario
- Medium
- Engineering practice
- Audit logs · Tenant authorization · Data governance
Estimated field mix
- Privacy engineering40%
- Backend30%
- Applied AI30%
Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.
Review it, then add it to your workspace.
The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.
Project context
A fictional software vendor receives billing, account-access, bug, and security reports. A prototype silently moves tickets based on vague model confidence. Replace it with bounded suggestions, deterministic safety rules, and an auditable review flow.
Setup prerequisites
- Create a synthetic support-ticket corpus with no real customer messages.
- Implement a deterministic local classifier double with success, malformed-output, and timeout modes.
Preceding work
Complete these dependencies, or supply their agreed outputs before taking this ticket.
- TRIAGE-101 · Define queue labels with examples and an unknown outcome
- TRIAGE-102 · Normalize inbound messages without discarding the original record
- TRIAGE-103 · Route declared security incidents to review before calling a classifier
- TRIAGE-104 · Accept only bounded suggestions from the classifier
- TRIAGE-105 · Show a suggested queue as an explicit agent action
- TRIAGE-106 · Discard a suggestion when the ticket changed while classification ran
Acceptance criteria
- Each correction links the original suggestion, selected queue, actor, time, and optional bounded reason.
- Corrections are append-only and readable only within the ticket tenant and permitted support role.
- No correction triggers external training or sends message text to a provider automatically.
Implementation constraints
- Treat corrections as operational feedback, not unquestionable ground truth.
Verification to include
- Correct one suggestion twice and inspect the complete ordered history.
- Attempt to read another tenant correction and confirm denial with no message text leakage.
Deliverables
- Correction audit model and scoped history endpoint
Rollout and recovery
Start collecting local operational feedback with retention controls; any later training export requires a separate governed workflow.
Value of the work
For the engineer: Practice constrained classification, human correction workflows, model versioning, and evaluation under ambiguous inputs.
For the team: Review whether automation saves triage effort while preserving queue ownership and safe escalation.
Evidence boundaries
Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.
Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.