AI agents in customer service: what to automate and what not to

How to use AI agents in customer service: which queries to automate, when to hand over to a person, how to use your own data and how to measure quality.

A customer service AI agent is an assistant that answers queries using the company’s documentation and systems, classifies requests and hands over to a person when it should not answer. Automate the frequent, clear-cut queries, not the sensitive ones. Designed well, it cuts response times and frees the team for the cases that genuinely need judgement.

What a customer service AI agent is

An AI agent is a system that understands natural language, consults authorised sources of information and carries out actions with limited permissions. In customer service that means three things: answering, classifying and handing over.

The difference from a traditional chatbot matters. A classic chatbot follows a decision tree and fails as soon as the customer steps off it. An agent understands the question even when it is badly worded, looks for the answer in your data and, if it cannot find it, says so and passes the case to a person.

Which queries are worth automating

Not everything should be automated. This table helps you decide:

Type of query Automate Why
Order or shipment status Yes The data is in your systems
Lead times, terms and hours Yes Stable, documented answer
Documents and invoices Yes Found and sent without judgement
Product questions Yes, within limits Only with verified information
Complaints Classify and hand over They need judgement and empathy
Legal or sensitive cases No Always a person

A good starting point is to count last quarter’s queries and group them by type. The most repeated ones with a clear answer are the first candidates.

When a person should step in

Handing over to a person is not a failure of the agent; it is part of its design. It should happen, at the very least, in these cases:

  • Missing information. The agent cannot find the answer in the authorised sources.
  • Low confidence. An answer is possible, but not certain.
  • Sensitive topic. Complaints, personal data, legal matters or key accounts.
  • Explicit request. The customer asks to speak to a person.

When it hands over, the agent should pass on the context: who the customer is, what they asked and what they have already been told. That way nobody has to start from scratch.

How the agent works with your own data

An agent’s quality depends on the data it consults. It works well when connected to:

  • Up-to-date product documentation, terms and policies.
  • The systems that hold customer data: orders, shipments, invoices, tickets.
  • The history of answers the team considers correct.

What the agent should not do is answer with generic knowledge from the internet. That is why it should cite the source of each answer and work with permissions: it may look up an order, but not change it without authorisation. We explain how to do this in private environments, without your data training third-party models, in AI and GDPR.

Which channel to start with

There is no need to open the agent on every channel at once. Each channel has its own pace and level of demand:

  • Email and forms. The best starting point. The agent can classify, draft a reply and, at first, let a person review it before it is sent.
  • Website chat. It requires answers within seconds and a very clear hand-off to a person. Open it once the agent has proved its accuracy on email.
  • Ticketing tool. The agent can summarise each case, suggest the category and propose a solution to the technician handling it.
  • Internal use. An agent that answers your own team from company documentation is a cautious way to start and to gather real cases.

Starting with one well-defined channel lets you measure, correct and build confidence before widening the scope.

How a customer service agent is implemented

An AI agent project follows the same steps as any process improvement:

  1. First conversation. Which queries arrive, through which channels and who handles them.
  2. Specifications. What the agent must do, what it must not do and when it must hand over.
  3. Development. Connection to documentation and systems, and design of the answers.
  4. Implementation and testing. Evaluation on real queries before opening it to customers.
  5. Go-live. First on one channel or one type of query, then the rest.
  6. Continuous evolution. Reviewing answers, adding sources and new cases.

The details are in how we work.

How to measure the agent’s quality

An AI agent is not judged by being live, but by how it answers. Define the indicators before starting and measure your current team’s starting point:

  • Average time to first response.
  • Share of queries resolved without human intervention.
  • Share of correct answers in a reviewed sample.
  • Hand-offs and the reason for each.
  • Customer satisfaction, if you already measure it.

Example: if the first response currently takes eight hours on average and, after go-live, 60% of queries are answered within minutes, the team can focus on the remaining 40%, which is where it adds the most.

Which mistakes to avoid

  • Opening the agent to every customer without testing it on real cases.
  • Connecting it to out-of-date documentation.
  • Hiding the option to speak to a person.
  • Not reviewing its answers after go-live.

There is more on what agents do well and where they fail in AI agents in the enterprise.

Where to start

Review last month’s queries and separate those with a clear, documented answer. That is the agent’s first job. If you would like to look at it with us, request a first conversation to get to know your company and see where we can help, or see how we approach AI agents.

Frequently asked questions

Which queries can an AI agent resolve?

Frequent ones with a clear answer: order status, lead times, terms, documents or product questions. Complex or sensitive complaints should go to a person.

Does the agent make answers up?

A well-designed agent answers only from the company's documentation and systems, cites the source and hands over to a person when it lacks enough information.

How does it fit with our tools?

It connects to email, the web form, the CRM or the ticketing tool, and logs every conversation where the company already works.

How do we know it answers well?

It is evaluated on real cases before going live and reviewed afterwards against indicators: correct answers, hand-offs, response time and customer satisfaction.

Let’s talk about your processes.

A first conversation to get to know your company and see where we can help.