- -68%First response time
- 94%Draft acceptance rate
- 0Unsourced claims in production
The challenge
Meridian's support team answered the same two hundred policy questions every week, each requiring a careful read of documents that changed monthly. Response times drifted past a day, and in a regulated setting a confidently wrong answer was a far worse outcome than a slow one.
What we did
- 01
Built retrieval over the verified policy corpus so every draft is grounded in a specific, current, citable passage.
- 02
Kept a human in the loop by design — the assistant drafts, an agent approves. It never sends unattended.
- 03
Defined explicit refusal behaviour: below a confidence threshold, or outside the corpus, it escalates instead of guessing.
- 04
Built an evaluation set of six hundred real historical tickets and measured accuracy against it before anything reached production.
The outcome
First response time dropped 68% while accuracy went up, because every answer now cites current policy. Agents accept 94% of drafts with light edits, and no unsourced claim has reached a customer.
“They pushed back hard on letting it reply unattended. In our sector that judgement was worth more than the feature would have been.”