AI support chatbots at SaaS companies fully resolve fewer than half of customer conversations. In an October 2026 test of 60 SaaS support chatbots, across 600 live conversations, bots fully resolved 42%. 35% ended with no answer at all. None of the 60 chatbots resolved all 10 questions, and the best resolved 9.
The data comes from the isalo support chatbot performance report, published October 29, 2026. [Placeholder: one sentence on how A6 Group relates to isalo, for example “isalo is an A6 Group company.”]
How the chatbots were tested
Between October 8 and 23, 2026, an AI agent played an anonymous customer on the public support pages of 60 SaaS companies. It stayed signed out, used no account and shared no personal details. Each chatbot got the same five topics, each asked two ways:
- Resetting a password
- Setting up alerts
- Fixing a 403 error
- Getting a SOC 2 report
- Cancelling a subscription
A separate AI evaluator graded every conversation against a written rubric. Each one ended in one of five outcomes: resolved, partial answer, routed to a person, gated by a login or contact form, or a dead end or wrong answer.
What share of chatbot conversations get resolved?
Bots fully resolved 42% of the 600 conversations. 23% got a partial answer. The rest ended without an answer: 7% were routed to a person, 17% were stopped by a login or contact form, and 11% hit a dead end or a wrong answer.
Results depended heavily on the topic. Bots handled everyday how-to questions best and business questions worst (see the chart below).
Cancellations and SOC 2 report requests were the hardest topics. They are also the questions that decide renewals and security reviews.
Why do AI support chatbots fail customers?
The failures were predictable. They clustered around questions that matter most to customers and buyers.
- Business questions get pushed away. Bots routed or gated business questions, such as cancellations and security reports, twice as often as how-to questions (34% vs. 17%).
- Security answers stay vague. In 46% of SOC 2 conversations, the bot never clearly said whether a report exists.
- Contract questions miss the key fact. Asked how to stop an annual contract from auto-renewing, 58% of bots gave neither the notice deadline nor where to find it.
- Links replace answers. In 8% of conversations, the customer got a link instead of an answer.
- Sales bots answer support questions. 9 out of 60 companies had a sales chatbot on their support page. Those bots resolved 33% of conversations, against 43% for the rest.
Does the chatbot platform make a difference?
Not much. Two companies using the same chatbot platform were 80 points apart in resolution rate. Every main platform showed a gap of 50 to 90 points between its strongest and weakest company. What the bot can find, and how the company sets it up, matter more than which vendor built it.
What do the best AI support chatbots do differently?
The strongest bots asked a clarifying question before answering, gave complete answers in the customer’s own words, and said plainly when information was missing. Few bots did the first: in only 8% of password conversations did the bot ask how the customer signs in, or whether they still can.
The report recommends five actions:
- Publish the answers your bot needs, including policies and security information.
- Sweat the small stuff, from repeated replies to unexplained jargon.
- Test the questions customers and buyers actually ask, in their words.
- Be transparent when the bot is an AI-powered knowledge base search.
- Re-test continuously, because bots and help content change.
The full findings, with examples of good and bad conversations, are free to read online.
Frequently asked questions
How accurate are AI customer support chatbots? In a 2026 test of 60 SaaS support chatbots, bots fully resolved 42% of 600 conversations. About 1 in 9 conversations (11%) ended in a dead end or a wrong answer.
What percentage of chatbot conversations end without an answer? 35%. Those conversations were routed to a person (7%), stopped by a login or contact form (17%), or ended in a dead end or wrong answer (11%).
Which customer questions do chatbots handle worst? Business questions. Bots resolved 24% of cancellation conversations and 33% of SOC 2 report requests, against 54% for setting up alerts.
Does the chatbot vendor determine how well a bot performs? No. Companies on the same platform were up to 80 points apart. The content the bot can draw on and the way it’s set up matter more.
How can I test my own support chatbot? Open your support page signed out, ask your bot the questions customers ask most, in their words, and ask each one twice in fresh chats. The isalo report includes a checklist for five common topics.
Read the full research
Your support chatbot is often the first conversation a customer or buyer has with your company. This study shows most bots answer the easy questions and stumble on the ones that decide renewals and deals.
Most of the results are free to read online, with charts and anonymized examples of actual bot replies. The complete 42-page report adds six annotated conversations, ten actions and a checklist to test your own bot. Get both from the isalo support chatbot performance 2027 research page.
Sources
- isalo, Support chatbot performance 2027, published October 29, 2026. Test window October 8 to 23, 2026.