How to Fix a Chatbot That Keeps Escalating Simple Questions

A report showing escalation volume broken down by reason

Troubleshooting / Ops Topics

September 22, 2026

Reading Time

6 minutes

A high escalation rate is not itself a problem, and the useful measure is what proportion of escalations the assistant could reasonably have handled.

Sampling and judging

Reviewing a sample of escalated conversations and judging whether the assistant could have answered gives a far more useful figure than the rate itself.

A sample of fifty is usually enough to establish the proportion and to identify the main causes at the same time.

Checking what agents did

Where agents resolve escalations quickly using information already documented, the escalation was almost certainly avoidable.

Where they consistently investigate or exercise judgement, the escalations are earning their cost and the threshold is correctly placed.

Separating policy escalations

Escalations triggered by safety, complaint or regulated topic rules are working as intended and must be excluded from the avoidable count.

Including them creates pressure to weaken exactly the rules that exist for good reason, which is the most dangerous outcome of this exercise.

Asking the agents

Agents receiving these escalations know immediately which ones the assistant should have handled, and asking them is faster than any sampling exercise.

Their answer also identifies the specific topics involved, which is the improvement list rather than a general finding.

Table of content

Cause One: Coverage Gaps

The largest category is usually questions the assistant escalates because nothing in the knowledge base answers them.

How to recognise it

The escalation reason is a lack of matching content rather than low confidence, and the agent answers from knowledge that exists somewhere but not in the assistant's sources.

These cluster around topics, which makes the improvement list short even when the escalation count is high.

Why it accumulates

New products, campaigns and policy changes generate questions continuously, and knowledge maintenance rarely keeps pace with them.

This means coverage gaps grow steadily unless someone reviews escalations regularly, which is a maintenance habit rather than a project.

The fix

Add the missing content, taking the answer from what agents actually say rather than writing something new, which is faster and more accurate.

This reduces escalation with no accuracy risk at all, which makes it the right place to start regardless of the other causes.

Cause Two: Intent Problems

Where the answer exists but the assistant did not recognise the question, the escalation is a classification problem rather than a knowledge one.

Overlapping intents

Two intents covering similar ground split confidence between them, so neither reaches the threshold and the conversation escalates despite being understandable.

This is the most common intent cause and it is fixed by consolidating rather than by adding more examples to both.

Unfamiliar phrasings

A recognisable request expressed in words the model has not seen produces low confidence, which is a training coverage gap rather than an intent design problem.

Adding the actual phrasings from escalated conversations resolves these directly and improves recognition generally.

Thin training data

An intent trained on a handful of examples produces low scores on legitimate variations, which escalates conversations the assistant could handle.

Reviewing which intents have the fewest examples usually identifies the ones responsible for a disproportionate share of escalations.

Cause Three: Over-Broad Escalation Rules

Rules written to catch sensitive topics frequently catch far more than intended, escalating ordinary questions that happen to contain a trigger word.

How to recognise it

The escalation reason is a rule rather than confidence, and reading the conversations shows a routine question containing a word the rule watches for.

Complaint and safety rules are the usual culprits, since they are deliberately written broadly and rarely revisited.

Narrowing carefully

These rules exist for good reason and narrowing them carries real risk, which means changes should be conservative and reviewed rather than optimised for volume.

Requiring more than one signal, rather than a single keyword, usually removes most false positives without weakening the protection.

What not to touch

Rules covering safety, welfare and explicit requests for a person should not be narrowed regardless of how many escalations they produce.

The cost of an unnecessary escalation on these is trivial against the cost of missing a real one.

Cause Four: Thresholds Set Too Conservatively

Where coverage and intents are sound, the remaining avoidable escalations are conversations the assistant understood but was not confident enough to answer.

Confirming this is the cause

The escalations show correct intent matching at confidence just below the threshold, and the agent gives the answer the assistant would have given.

If this pattern is not visible, the threshold is not the problem and lowering it will introduce errors rather than reduce escalations usefully.

Relaxing increm€entally

Lower the threshold in small steps with a measurement period between them, checking accuracy rather than only escalation volume after each change.

Changing thresholds and content simultaneously makes the result uninterpretable, which is how tuning becomes guesswork.

Checking both error types

After lowering a threshold, sample high-confidence conversations for wrong answers, since a reduction in escalation that introduces confident errors is not an improvement.

Where accuracy holds, the change was correct and the next step can be taken.

Work Through the Causes in Order

Address coverage gaps first, then intent problems, then over-broad rules, and only then thresholds, since the risk increases at each step.

Why the order matters

Coverage and intent fixes reduce escalation with no accuracy cost, whereas threshold changes trade one error type for another.

Starting with thresholds is common and produces a system that escalates less and answers worse, which is the wrong outcome.

Measuring between steps

Each fix should be followed by a measurement period, since the causes interact and closing a coverage gap frequently reduces the apparent threshold problem.

Working through them sequentially takes longer and produces a system whose behaviour is understood.

Re-sampling afterwards

Repeating the avoidable escalation sample after the changes confirms whether the work reduced what it was meant to reduce.

This also identifies whether the remaining escalations are now genuinely necessary, which is the point at which to stop.

How Chatboq Supports This

Chatboq records an escalation reason on every handoff, reports rate by reason and topic, and separates policy escalations from performance ones.

Reasons recorded automatically

Every escalation is tagged with whether it was triggered by coverage, confidence, a rule or an explicit request, which is the classification this diagnosis depends on.

Because it is recorded at the point of escalation, the analysis is a report rather than a manual review exercise.

Policy escalations protected

Safety, complaint and regulated topic escalations are reported separately and excluded from reduction targets, which prevents them being weakened under pressure.

This separation is what makes it safe to set a target on the remainder.

Accuracy checks alongside

Sampling of high-confidence conversations is available alongside escalation reporting, so a threshold change can be evaluated on both error types.

That pairing is what distinguishes a genuine improvement from a reduction achieved by answering things wrongly.

Frequently AskedQuestions

By recording an escalation reason on every handoff, reporting by reason and topic, separating protected policy escalations, and pairing this with accuracy sampling.

Safety, welfare and explicit requests for a person, where the cost of an unnecessary escalation is trivial against the cost of missing a real case.

Overlapping intents splitting probability between them, unfamiliar phrasings the model has not seen, and intents trained on very few examples.

Not first. Threshold changes trade escalations for wrong answers, whereas closing coverage gaps and fixing intents reduces escalation with no accuracy cost.

Sample escalated conversations and check whether agents resolved them quickly using documented information, which indicates the assistant could have answered.

Usually coverage gaps where nothing answers the question, intent problems where it was not recognised, over-broad escalation rules, or thresholds set too conservatively.

Sampling and judging

Reviewing a sample of escalated conversations and judging whether the assistant could have answered gives a far more useful figure than the rate itself.

A sample of fifty is usually enough to establish the proportion and to identify the main causes at the same time.

Checking what agents did

Where agents resolve escalations quickly using information already documented, the escalation was almost certainly avoidable.

Where they consistently investigate or exercise judgement, the escalations are earning their cost and the threshold is correctly placed.

Separating policy escalations

Escalations triggered by safety, complaint or regulated topic rules are working as intended and must be excluded from the avoidable count.

Including them creates pressure to weaken exactly the rules that exist for good reason, which is the most dangerous outcome of this exercise.

Asking the agents

Agents receiving these escalations know immediately which ones the assistant should have handled, and asking them is faster than any sampling exercise.

Their answer also identifies the specific topics involved, which is the improvement list rather than a general finding.

Table of content

Cause One: Coverage Gaps

The largest category is usually questions the assistant escalates because nothing in the knowledge base answers them.

How to recognise it

The escalation reason is a lack of matching content rather than low confidence, and the agent answers from knowledge that exists somewhere but not in the assistant's sources.

These cluster around topics, which makes the improvement list short even when the escalation count is high.

Why it accumulates

New products, campaigns and policy changes generate questions continuously, and knowledge maintenance rarely keeps pace with them.

This means coverage gaps grow steadily unless someone reviews escalations regularly, which is a maintenance habit rather than a project.

The fix

Add the missing content, taking the answer from what agents actually say rather than writing something new, which is faster and more accurate.

This reduces escalation with no accuracy risk at all, which makes it the right place to start regardless of the other causes.

Cause Two: Intent Problems

Where the answer exists but the assistant did not recognise the question, the escalation is a classification problem rather than a knowledge one.

Overlapping intents

Two intents covering similar ground split confidence between them, so neither reaches the threshold and the conversation escalates despite being understandable.

This is the most common intent cause and it is fixed by consolidating rather than by adding more examples to both.

Unfamiliar phrasings

A recognisable request expressed in words the model has not seen produces low confidence, which is a training coverage gap rather than an intent design problem.

Adding the actual phrasings from escalated conversations resolves these directly and improves recognition generally.

Thin training data

An intent trained on a handful of examples produces low scores on legitimate variations, which escalates conversations the assistant could handle.

Reviewing which intents have the fewest examples usually identifies the ones responsible for a disproportionate share of escalations.

Cause Three: Over-Broad Escalation Rules

Rules written to catch sensitive topics frequently catch far more than intended, escalating ordinary questions that happen to contain a trigger word.

How to recognise it

The escalation reason is a rule rather than confidence, and reading the conversations shows a routine question containing a word the rule watches for.

Complaint and safety rules are the usual culprits, since they are deliberately written broadly and rarely revisited.

Narrowing carefully

These rules exist for good reason and narrowing them carries real risk, which means changes should be conservative and reviewed rather than optimised for volume.

Requiring more than one signal, rather than a single keyword, usually removes most false positives without weakening the protection.

What not to touch

Rules covering safety, welfare and explicit requests for a person should not be narrowed regardless of how many escalations they produce.

The cost of an unnecessary escalation on these is trivial against the cost of missing a real one.

Cause Four: Thresholds Set Too Conservatively

Where coverage and intents are sound, the remaining avoidable escalations are conversations the assistant understood but was not confident enough to answer.

Confirming this is the cause

The escalations show correct intent matching at confidence just below the threshold, and the agent gives the answer the assistant would have given.

If this pattern is not visible, the threshold is not the problem and lowering it will introduce errors rather than reduce escalations usefully.

Relaxing increm€entally

Lower the threshold in small steps with a measurement period between them, checking accuracy rather than only escalation volume after each change.

Changing thresholds and content simultaneously makes the result uninterpretable, which is how tuning becomes guesswork.

Checking both error types

After lowering a threshold, sample high-confidence conversations for wrong answers, since a reduction in escalation that introduces confident errors is not an improvement.

Where accuracy holds, the change was correct and the next step can be taken.

Work Through the Causes in Order

Address coverage gaps first, then intent problems, then over-broad rules, and only then thresholds, since the risk increases at each step.

Why the order matters

Coverage and intent fixes reduce escalation with no accuracy cost, whereas threshold changes trade one error type for another.

Starting with thresholds is common and produces a system that escalates less and answers worse, which is the wrong outcome.

Measuring between steps

Each fix should be followed by a measurement period, since the causes interact and closing a coverage gap frequently reduces the apparent threshold problem.

Working through them sequentially takes longer and produces a system whose behaviour is understood.

Re-sampling afterwards

Repeating the avoidable escalation sample after the changes confirms whether the work reduced what it was meant to reduce.

This also identifies whether the remaining escalations are now genuinely necessary, which is the point at which to stop.

How Chatboq Supports This

Chatboq records an escalation reason on every handoff, reports rate by reason and topic, and separates policy escalations from performance ones.

Reasons recorded automatically

Every escalation is tagged with whether it was triggered by coverage, confidence, a rule or an explicit request, which is the classification this diagnosis depends on.

Because it is recorded at the point of escalation, the analysis is a report rather than a manual review exercise.

Policy escalations protected

Safety, complaint and regulated topic escalations are reported separately and excluded from reduction targets, which prevents them being weakened under pressure.

This separation is what makes it safe to set a target on the remainder.

Accuracy checks alongside

Sampling of high-confidence conversations is available alongside escalation reporting, so a threshold change can be evaluated on both error types.

That pairing is what distinguishes a genuine improvement from a reduction achieved by answering things wrongly.

Frequently AskedQuestions

By recording an escalation reason on every handoff, reporting by reason and topic, separating protected policy escalations, and pairing this with accuracy sampling.

Safety, welfare and explicit requests for a person, where the cost of an unnecessary escalation is trivial against the cost of missing a real case.

Overlapping intents splitting probability between them, unfamiliar phrasings the model has not seen, and intents trained on very few examples.

Not first. Threshold changes trade escalations for wrong answers, whereas closing coverage gaps and fixing intents reduces escalation with no accuracy cost.

Sample escalated conversations and check whether agents resolved them quickly using documented information, which indicates the assistant could have answered.

Usually coverage gaps where nothing answers the question, intent problems where it was not recognised, over-broad escalation rules, or thresholds set too conservatively.

How to Fix a Chatbot That Keeps Escalating Simple Questions

Sampling and judging

Checking what agents did

Separating policy escalations

Asking the agents

Cause One: Coverage Gaps

How to recognise it

Why it accumulates

The fix

Cause Two: Intent Problems

Overlapping intents

Unfamiliar phrasings

Thin training data

Cause Three: Over-Broad Escalation Rules

How to recognise it

Narrowing carefully

What not to touch

Cause Four: Thresholds Set Too Conservatively

Confirming this is the cause

Relaxing increm€entally

Checking both error types

Work Through the Causes in Order

Why the order matters

Measuring between steps

Re-sampling afterwards

How Chatboq Supports This

Reasons recorded automatically

Policy escalations protected

Accuracy checks alongside

Frequently AskedQuestions

How does Chatboq help?

Which escalation rules should never be narrowed?

What causes low confidence on questions the bot knows?

Should I just lower the threshold?

How do I know if an escalation was avoidable?

Why does my chatbot escalate simple questions?

How to Fix a Chatbot That Keeps Escalating Simple Questions

Sampling and judging

Checking what agents did

Separating policy escalations

Asking the agents

Cause One: Coverage Gaps

How to recognise it

Why it accumulates

The fix

Cause Two: Intent Problems

Overlapping intents

Unfamiliar phrasings

Thin training data

Cause Three: Over-Broad Escalation Rules

How to recognise it

Narrowing carefully

What not to touch

Cause Four: Thresholds Set Too Conservatively

Confirming this is the cause

Relaxing increm€entally

Checking both error types

Work Through the Causes in Order

Why the order matters

Measuring between steps

Re-sampling afterwards

How Chatboq Supports This

Reasons recorded automatically

Policy escalations protected

Accuracy checks alongside

Frequently AskedQuestions

How does Chatboq help?

Which escalation rules should never be narrowed?

What causes low confidence on questions the bot knows?

Should I just lower the threshold?

How do I know if an escalation was avoidable?

Why does my chatbot escalate simple questions?

Related Articles