What Is a Chatbot Confidence Score Threshold? Explained Simply

The threshold is the line the score must exceed for the assistant to act on its interpretation.

Above the line

When the score exceeds the threshold, the assistant treats the interpretation as correct and answers accordingly.

This is the intended behaviour for the large majority of messages in a well-configured deployment.

Below the line

When the score falls short, the assistant does not answer and instead falls back, which usually means asking again or offering a person.

This is the system correctly declining to guess rather than failing, though customers experience it as a failure.

The middle band

Many deployments use two lines, answering above the higher one and asking a clarifying question between them.

This recovers messages that were probably understood correctly without acting on an uncertain interpretation.

What happens after the threshold

The threshold decides whether to act on the understanding, and separate rules decide whether to involve a person, which is a different setting.

Conflating the two is common and leads to threshold changes when an escalation policy change was needed.

What customers experience

Customers do not see the score or the threshold, only whether the assistant answered or said it did not understand.

Framing the setting in those terms makes it a service decision rather than a technical one.

A high threshold produces frequent fallbacks and a low one produces confident wrong answers, and both are costly in different ways.

Set too high

The assistant falls back on messages it understood perfectly well, which frustrates customers and increases the load on agents unnecessarily.

This is the visible failure, which is why it tends to be over-corrected.

Set too low

The assistant answers messages it did not understand, producing fluent responses to questions nobody asked.

This is the invisible failure, since it never appears in fallback statistics and is only found by reading conversations.

Why low is more dangerous

A fallback is annoying and honest, whereas a confident wrong answer can lead a customer to act on incorrect information.

This asymmetry is why erring toward a higher threshold is the safer default, particularly on consequential topics.

Why One Threshold Is Not Enough

The cost of a wrong answer varies enormously by topic, which means a single setting will be wrong somewhere.

Low-consequence questions

Getting opening hours slightly wrong costs a customer a few minutes, which means a permissive threshold is reasonable there.

These topics also tend to have distinctive vocabulary and score highly anyway.

High-consequence questions

Eligibility, pricing, safety and anything with financial or legal effect warrant a much higher bar, since a wrong answer causes real harm.

Many deployments set these to escalate rather than answer regardless of confidence.

Grouping rather than individual settings

Setting a threshold for every topic becomes unmaintainable, so grouping topics into two or three consequence bands is the practical approach.

Three bands covering routine, moderate and high consequence suits most deployments.

Most deployments should leave the default alone initially and change it only with evidence.

Not on day one

A new deployment has no data showing where the line should sit, which makes early adjustment guesswork with extra steps.

Starting conservatively and relying on fallback and escalation to protect customers is the right approach while data accumulates.

Not to fix a different problem

Lowering the threshold to reduce fallbacks caused by missing content or overlapping topics will not help, since there is nothing correct to match.

Checking what the fallbacks actually contain, before adjusting anything, saves considerable wasted effort.

When adjustment is warranted

Where fallbacks are occurring on messages the assistant clearly understood, and accuracy on similar messages is good, the threshold is the constraint.

At that point adjustment is appropriate, in small steps with accuracy checked after each.

The threshold is a business risk decision rather than a technical one, which means it belongs with whoever owns the customer experience.

A judgement about acceptable error

Deciding how confident the assistant must be before answering a pricing question is a business judgement about acceptable risk.

Framing it that way makes the decision reviewable by people who understand the consequences rather than only the mechanics.

Documenting the reasoning

Recording why each band was set where it was prevents someone later adjusting it without understanding the assessment behind it.

This also makes the configuration reviewable by compliance or leadership where that matters.

Reviewing after changes

Adding new topics changes how scores distribute across existing ones, which can quietly alter behaviour nobody touched.

Reviewing thresholds after significant changes catches this before it appears as an unexplained shift in fallback rate.

Chatboq presents the threshold as a consequence-based setting with sensible defaults, rather than as a raw technical value.

Grouped by consequence

Topics are grouped into consequence bands with a threshold per band, which is a decision a support manager can make and review.

Sensible defaults apply from the start, so a new deployment does not require this decision before launch.

A clarifying band by default

Messages falling just short of the answering threshold produce a short clarifying question rather than a fallback, which recovers most of them.

This is what makes a conservative threshold tolerable rather than frustrating.

Evidence before adjustment

Score distributions and fallback contents are reported, so a change is made from evidence rather than from an impression that the assistant answers too rarely.

For the detail of how to tune the number, and how escalation policy differs from classification confidence, those are separate deeper topics.