Context, scope and the problem I actually solved

Support volume grew 50% a year. Headcount did not.

That is a straightforward margin problem before it is a design problem: every additional thousand tickets a month either degrades service levels or adds people. Leadership framed the goal as letting customer success scale volume without scaling headcount linearly, while improving response times rather than trading them away.

Loss runs were chosen as the first target because they were the cleanest mechanical case in the queue, not because they were the most painful. About 1,500 a month, roughly 10% of all tickets, spiking to 2,400 in a heavy month. Five minutes of agent time each, spread across seven steps in tools that did not talk to each other. Automating them removed an estimated 700 to 1,000 tickets a month from the queue on its own, and more importantly it created a template for everything after it.

Sole designer, discovery through launch and two years of expansion

I owned the interaction model, the runbook pattern, the trust boundary, the panel architecture, and the microcopy. I worked directly with one machine learning engineer, the platform engineers, and a PM, and I reported to the Head of Design and the Chief Product Officer.

I also helped define the success criteria the program was measured on, which is not a thing designers are usually invited to do and is the part of this project I would point at first. It is also the part I got wrong, and the reflection at the end is about that.

Two things I did not own: the model itself and the underlying pipeline architecture. I shaped what the model was asked to produce and what the product did when it was wrong, which is a different job from building it, and I try to be precise about the difference.

The real problem was not automation. It was whether to trust one specific proposal.

The assigned problem was "automate loss runs." The actual problem was narrower and harder to state: an agent needs to decide whether to trust a specific machine-generated proposal, in under a few seconds, without leaving the ticket, and the cost of trusting a wrong one is a broker receiving an incorrect document.

That framing is testable in a way the brief was not. It names who is affected, where the decision happens, what has to fit in the available time and space, and what failure looks like. Every design decision that follows either serves that sentence or it does not.

Three constraints were fixed going in. The panel width, which I have already spent a section on. The tool boundary: agents work in Zendesk and any solution that moves them out of it spends the time it was built to save. And a limited internal model, where engineering did not want to spend unnecessary calls on work that might fail anyway.

The last one shaped the interface more than it looks like it should have. It is why the runbook is a stepper rather than a single generated block: decomposing the work into discrete verifiable steps meant a failure at step two did not waste the calls for steps three and four, and it gave the agent somewhere specific to look when something was off. A structural constraint from engineering became the organizing logic of the UI.

The five minutes was never thinking time. It was seven context switches.

I did not come to this cold. I had spent years on the internal tools these same agents used, including the quoting redesign, which meant I had the relationships before I had the project. I had spent time in the Provo office with the customer success team and I ran the testing sessions myself. When I say the team would not close tabs or overlays, that is observed behaviour from watching them work, not a usability heuristic.

The decisive finding was about where the time actually went. Five minutes per loss run is not five minutes of thinking. It is seven context switches, each cheap on its own, and the reason the task felt heavy was the switching rather than the judgment. That reframed the target. The product did not need to make agents faster at the task. It needed to collapse the task into one place, and then the speed followed.

The interaction model

What the AI claims sits in a box. What the human does does not.

The interface makes a visual distinction between assertions and workspace. Everything the model has concluded lives in a filled, collapsible grey card headed "Step 1: Verify Request Details." Everything the agent does sits on plain white. Contain what the AI claims, leave the human's area uncontained. The heading verb is "Verify" rather than "Review," because review is passive and verify names an action with a right answer.

Image: the containment study. Winning layout beside two rejected alternatives.

Confidence scores existed, and I kept them out of the panel

The system produced them. copilot_failed, language_confidence__high, sentiment_confidence__high all existed as ticket tags. I chose not to surface them to agents. A confidence score invites people to trust the machine's self-report instead of the evidence, and a model that is confidently wrong is exactly the failure mode we already had. I would rather an agent glance at a source snippet than read a number the model assigned itself.

Autonomy is a property of the situation, not a setting on the product

Full automation only fires when the request carries a policy number and all of the organization's policies sit on one account, when policies remain separable if renewals transfer between agencies, and when the requester is the broker of record or identifiably from the same agency. Roughly 55 to 70% of loss run requests qualify. Everything else stays supervised. Writing those criteria down is what let us expand coverage later without relitigating trust each time.

Where I lost one: auto-selecting intent

The PM wanted intent and work type auto-selected rather than left for the agent to confirm, because he was not confident the context would survive otherwise. I would have preferred confirmation as an explicit act. He was closer to the failure cases than I was and I took the call. It is the decision I am least settled on and I would look at it again with the data we now have.

Designing the failure path as a real path, not an error state

The 80% acceptance rate was the number at launch rather than something we climbed to, which meant the interesting design work was in the remainder. It split cleanly in two. Some were account matching problems, where the request could not be tied confidently to one account, and those got a dedicated search and disambiguation flow. The rest had a claim attached, which needs a specialist rather than better matching, so those route out: a dropdown of specialized users and an "Assign to agent" action.

A tool that works four times out of five and dead-ends on the fifth teaches people to check it manually every time. Designing the fifth case as a route rather than a wall is most of why agents kept using the product.

What would not have happened without me: the upstream PDF fix

Nobody else was looking outside the interface for the cause of an interface problem, and I was the only person who owned both the exporter and the panel.

The hover pattern crossing into claims is the second. I was also the designer on the claims workbench, which had the identical shape: extracted values from long invoices that nobody would open a document to confirm. I took it to that team and their PMs as reuse of something that already existed and worked, which was an easier conversation than a new pattern would have been, though claims had never used Copilot at all so I had to demonstrate the product before I could demonstrate the pattern. Labelled fields and hover to source now run in both verticals. The broader runbook and containment model went further: three teams, including underwriting workbench and claims, built their own AI features on it.

I worked the trust boundary jointly with the machine learning engineer, and the eligibility criteria came out of that back and forth rather than from either side alone.

Image: the same hover pattern running on a claims invoice.