Enterprise AI
Seven questions to answer before you deploy AI customer service
Jesse Huang Published
When we talk to a company about AI customer service, the first question is rarely “which model is best”. It is “when do you want it to stay quiet”. That is not a joke. Looking back at rollouts that quietly faded, the cause was seldom the technology. It was that the seven questions below were never answered before the contract was signed.
1. Which questions must the AI not answer?
Everyone starts by listing what they hope the AI can answer. What actually decides the outcome is the other list: the topics it must not touch. Refund amounts, liability, anything resembling legal or medical advice, conversations that have already escalated into complaints. Hand these to an AI and nobody remembers the correct answers, while the wrong ones get screenshotted and forwarded.
Draw the “do not answer” boundary first, then talk about capability. An agent that knows its limits is worth more than one that will answer anything.
2. Where do the answers come from, and who keeps them current?
Every sentence an AI agent produces has a source: FAQs, product documentation, internal procedures, past conversations. The problem is never the absence of sources. It is that nobody owns keeping them correct.
Prices change, promotions end, return policies get revised. Without one person who explicitly owns the answer sources and an update cadence, the agent starts confidently quoting outdated information in its second month. It is the most common failure we see, and the most embarrassing, because everything looks fine from the outside.
3. How is personal data in the conversations handled?
Customer service conversations are full of personal data: names, phone numbers, orders, addresses, sometimes national ID numbers. The question to answer before rollout is not “is it secure”, which cannot be verified. It is a set of specific ones: where the notice required by Taiwan’s Personal Data Protection Act lives; whether transcripts are stored inside or outside Taiwan; for how long; who inside the company can read them; and whether the model vendor uses the conversations for training.
In Taiwan, the LINE official account is the most common customer service channel. When you connect it, read LINE’s data terms alongside the model vendor’s terms. Reading only one side is not enough.
4. When the AI is wrong, who takes over?
The AI will be wrong. That is a premise, not a risk, and the workflow should treat it as something that will certainly happen.
What needs to be decided is the handover trigger: the customer asks for a person, the tone turns hostile, the AI’s confidence is low, the same question is asked twice. The handover must carry the conversation so far, so the customer never has to repeat themselves. And there has to be a commitment on how quickly a person replies. Without this design, AI customer service simply exhausts the customer’s patience at an earlier stage.
5. How will you know it works?
“Customers seem happy” is not a metric. Define measurable things before launch: the share of conversations resolved without a human, the handover rate, time to first reply, the share of customers who write again about the same issue, and customer self-ratings.
More importantly, measure the current state first. If nobody knows what those numbers are for the human team today, there is no way to tell after launch whether things got better or worse. A project without a baseline closes with a report written in feelings.
6. Should it connect to existing systems?
CRM, ticketing, the order system, the LINE official account, phone IVR. Every integration expands what the AI can do, and expands the blast radius when it gets something wrong.
Our recommendation for phase one: read, do not write. Let the AI look up an order status; do not let it modify the order. Once it has proven it reads accurately, discuss write access. This is not caution for its own sake. It is containing the blast radius.
7. How big should the first month be?
One channel, one category of questions, one time window. That is a reasonable pilot.
Write down the exit conditions at the same time: handover rate above the expected level, complaint volume rising, one category of question consistently answered wrong. Any of these means scaling back. The goal of the first month is to learn how this company’s customers actually ask questions, not to cut cost. Make cost the first-month goal and you usually get neither.
After the answers
A company that has answered all seven chooses tools quickly, because it already knows what it wants, what it does not, and how it will judge the result.
Our own Customer Service AI OS is in early access. We would rather work through these seven questions with you first and then decide whether it fits, not the other way round. If you are evaluating, start with a real problem.