Explain routing delays using metadata instead of customer messages
Support leads need to diagnose slow suggestions, while logs currently include full complaint text. Add safe operational measures and an immediate suggestion kill switch.
- Focused work estimate
- 1h 45m + prerequisites
- Priority in the scenario
- Medium
- Engineering practice
- Observability · Privacy · Operational controls
Estimated field mix
- Applied AI40%
- Site reliability30%
- Privacy engineering30%
Field percentages are editorial estimates of the ticket's engineering focus. They total 100%; they are not measured time, proficiency scores, or ownership evidence.
Review it, then add it to your workspace.
The board opens an editable draft; nothing is saved until you confirm it. Sign-in and workspace permissions apply, and Demo boards remain ephemeral.
Project context
A fictional software vendor receives billing, account-access, bug, and security reports. A prototype silently moves tickets based on vague model confidence. Replace it with bounded suggestions, deterministic safety rules, and an auditable review flow.
Setup prerequisites
- Create a synthetic support-ticket corpus with no real customer messages.
- Implement a deterministic local classifier double with success, malformed-output, and timeout modes.
Preceding work
Complete these dependencies, or supply their agreed outputs before taking this ticket.
- TRIAGE-101 · Define queue labels with examples and an unknown outcome
- TRIAGE-102 · Normalize inbound messages without discarding the original record
- TRIAGE-103 · Route declared security incidents to review before calling a classifier
- TRIAGE-104 · Accept only bounded suggestions from the classifier
- TRIAGE-107 · Keep provider outages from blocking the support inbox
- TRIAGE-105 · Show a suggested queue as an explicit agent action
- TRIAGE-106 · Discard a suggestion when the ticket changed while classification ran
- TRIAGE-108 · Record corrections without turning agent clicks into automatic training data
- TRIAGE-109 · Compare two routing revisions on the same ambiguous-ticket set
Acceptance criteria
- Metrics expose queue delay, provider duration, timeout count, validation failures, and manual-review count without message bodies.
- Audit metadata includes trace ID and exact provider/template/schema versions.
- Disabling suggestions stops new provider calls while tickets and existing audit history remain accessible.
Implementation constraints
- Represent unavailable provider cost as unavailable, not zero.
- Keep privileged access to source messages outside generic analytics.
Verification to include
- Run a delayed provider fixture and trace intake-to-review timing from metadata.
- Toggle the kill switch with queued work and confirm no new calls and no lost tickets.
Deliverables
- Safe operations dashboard data and disable/recovery runbook
Rollout and recovery
Enable metrics before activating suggestions; use the kill switch for unexpected routing changes while retaining manual service.
Value of the work
For the engineer: Practice constrained classification, human correction workflows, model versioning, and evaluation under ambiguous inputs.
For the team: Review whether automation saves triage effort while preserving queue ownership and safe escalation.
Evidence boundaries
Outcome Evidence: Tests, patches, and runbooks are requested deliverables. They become Outcome Evidence only through a qualified Mission and immutable Evidence IDs.
Ownership Evidence: Independent adaptation must be observed under a declared verification policy and cite immutable Evidence IDs. Completing a planning ticket establishes no Ownership Evidence.