Illustrative Scenario
Support triage with human handoff
Illustrative scenario · not a client engagement
Current situation
A small support team handles one email queue. A large share of the questions repeat, and the answers already exist in internal documents that are hard to search quickly.
The flow we propose
One queue is classified, an approved knowledge set is searched, and an answer is drafted with citations to the documents it came from. Anything below the confidence threshold, account-specific or policy-sensitive is escalated instead of answered.
From a real run
In this reference flow, a drafted answer is only produced from the selected documents it can cite; when there isn't enough grounding, the system abstains and routes the question for review instead of guessing. Every answer also waits for a person's approval before anything is sent.
Captured from our synthetic reference build — not a client result.View full size →
How it works today
Each message is read, someone searches shared documents for the relevant policy, writes an answer from memory when the search fails, and forwards anything account-specific to a colleague.
Human control points
- Every drafted answer is reviewed by an agent before it is sent during the pilot.
- The system does not answer and passes the case to a person when it cannot cite an approved source.
- Account-changing requests are never actioned by the system, only routed.
Failure and delay points
- Answers vary between team members because the source is not consistently checked.
- Repeat questions consume the same effort every time.
- Account-specific requests sit in the queue until the right person is free.
What the pilot measures
- First-response time against the agreed baseline
- Citation correctness on an agreed test set
- Rate of answers with no supporting source
- Escalation accuracy on cases that should have been escalated
Technical detail and assumptions
More evidence
Systems and data
One email queue and an approved set of documents the client agrees in advance. Nothing outside that set is retrieved, and the drafted answer carries the citation list with it.
Baseline assumptions
- Assumed volume: about 300 messages a month in one queue. Assumption for scoping only.
- Assumed repeat share: roughly half the questions are variations of known topics. This is an assumption to be tested during the pilot, not a measured baseline.
- Assumed knowledge set size: around 40 documents, kept deliberately small so citation quality can be judged.
Pilot scope
One queue, one approved knowledge set, drafts only, and a fixed confidence and policy rule for escalation. No autonomous sending, no account changes, no additional channels.
Risks, exclusions and fallback
- If retrieval quality drops below the agreed threshold, drafting is disabled and the queue returns to manual handling.
- Out-of-scope topics are escalated rather than answered from general knowledge.
- Availability of the underlying model provider is a dependency outside this workflow.
What moving to production would require
An owner for the knowledge set with a review cadence, an agreed escalation policy, logging and retention boundaries for message content, an evaluation set that is rerun after changes, and a safe-disable switch.