Retrieval-augmented support copilot for a customer-facing team
AI customer-support assistant trained on the client's internal knowledge base. Deflects routine tickets, drafts replies for complex ones, escalates with full context.
Client:B2B SaaS customer (anonymized)
StatusIn-Beta
PatternDrafting copilot
GuardrailsAudit + eval + human-in-loop
StackClaude / GPT + retrieval + structured output
The problem
A B2B SaaS customer-facing team was answering the same kinds of tickets repeatedly. Generic ChatGPT wrappers had been tried and abandoned — hallucinations, no context on the actual customer, no audit trail, no path to improve. The team wanted AI that augmented agents, not replaced them.
What we built
- Retrieval pipeline over the customer's knowledge base and historical ticket corpus.
- Drafting copilot — AI proposes a reply, agent reviews and edits before send.
- Structured output schemas — critical fields (account ID, order number, refund amount) validated against source.
- Eval harness with golden dataset, automated grading, and a baseline score every release must beat.
- Guardrails — PII redaction, response-policy enforcement, prompt-injection defense.
- Observability — every prompt, response, latency, and cost logged for analysis.
- Continuous improvement loop — production traces feed back into evals.
Engineering philosophy
Evals before code. Retrieval before model. Structured outputs over free-text where downstream code depends on the answer. Humans stay in the loop on every escalation. Critical fields are never trusted from the model alone — they're validated against source data.
Outcomes
- First-response time measurably cut.
- Audit trail on every AI reply, exportable for compliance review.
- Eval scores stay green release-over-release — regressions caught before customers see them.
See our AI development service.