How to Fix a Chatbot That Doesn't Understand Follow-Up Questions

Pronouns and demonstratives such as it, that one and the second require the system to know what they refer to, which many configurations never attempt.

How to recognise it

The follow-up contains a pronoun or a reference, the relevant subject was mentioned earlier, and the response addresses something else entirely or falls back.

This is the most common follow-up failure and the most visible to customers, since the reference seems obvious to them.

Why it happens

Reference resolution requires the system to track what was recently discussed and match a pronoun against it, which is a dialogue management capability rather than a language one.

Systems built as a series of independent intents have nowhere for that tracking to live.

The fix

Retain the most recently discussed entities and resolve references against them, confirming the interpretation where more than one candidate exists.

Confirming rather than guessing is important, since resolving a pronoun to the wrong subject produces a confidently wrong answer.

Customers omit words they consider obvious, asking how much or by when, and a system expecting complete questions will not recognise these.

How to recognise it

The follow-up is very short, grammatically incomplete and perfectly clear to a human reading the conversation, and the system falls back on it.

Two or three word messages following a substantive answer are almost always ellipsis rather than a new topic.

Why short messages fail

Classification on a short message has very little signal to work with, so confidence is low and the fallback threshold catches it even where the meaning was clear.

This is why follow-ups fail more often than opening questions despite being easier for a person to understand.

The fix

Interpret short messages in the context of the previous exchange rather than in isolation, treating them as continuations of the active topic by default.

Where that produces low confidence, asking a short clarifying question is considerably better than a fallback.

Where context is discarded as soon as an intent completes, the follow-up arrives with nothing to build on even though it was asked seconds later.

How to recognise it

The follow-up fails immediately after a successfully completed answer, which is the point at which many systems reset their state.

Checking what was retained at that moment confirms it, since the state will be empty rather than incorrect.

Why it is configured that way

Clearing aggressively prevents stale context being applied to a new topic, which is a real problem, so the setting is usually a deliberate over-correction.

The result is that the system avoids one failure mode by guaranteeing another.

The fix

Retain the topic and recently discussed entities for a period after an intent completes, clearing them on a genuine topic change rather than on completion.

Identity and account context should persist for the whole session regardless, since these almost never become stale within one conversation.

A follow-up classified as a fresh request restarts the conversation, which produces a technically correct answer to the wrong question.

How to recognise it

The system responds with an opening-style answer, or asks for information the customer already provided, which indicates it started again rather than continued.

Checking the matched intent confirms it, since the intent will be a top-level one rather than a continuation.

Distinguishing continuation from change

The system needs to judge whether a message continues the current topic or starts a new one, and defaulting to new is what causes this failure.

Defaulting to continuation, and detecting genuine topic changes explicitly, produces better behaviour for most conversations.

Handling side questions

A customer mid-flow who asks something unrelated expects an answer and then to continue, not to have the flow abandoned.

Supporting suspension and resumption is what makes this work, and it is a dialogue management feature rather than an intent one.

Follow-up failures do not appear in single-message testing, which is why they reach production so consistently.

Building follow-up test cases

Every significant flow should have test cases covering a pronoun reference, an elliptical question and a side question, since these are the three common patterns.

Writing them takes minutes and they catch the majority of follow-up regressions before release.

Replaying real transcripts

Real conversations contain follow-ups nobody would invent, which makes transcript replay considerably more revealing than written test cases.

Conversations that previously failed are the most valuable to replay after any change.

Measuring the failures

Fallbacks occurring on the second or later turn, rather than the first, are a direct measure of follow-up handling quality.

Tracking this separately from overall fallback rate makes the problem visible rather than averaged away.

Chatboq retains topic and entity context across turns, resolves references, defaults short messages to continuation and clears state on genuine topic change.

Context retained deliberately

Recently discussed entities and the active topic persist after an intent completes, so an immediate follow-up has something to build on.

Clearing happens on a detected topic change rather than on completion, which avoids the aggressive reset that causes most of these failures.

References resolved with confirmation

Pronouns and demonstratives resolve against recently discussed entities, with a short confirmation where more than one candidate exists.

Confirming rather than guessing prevents the confidently wrong answers that reference resolution can otherwise produce.

Reporting late-turn fallbacks

Fallbacks occurring after the first turn are reported separately, which surfaces follow-up handling problems that a blended fallback rate hides.

Those conversations are linked directly, so the specific phrasings can be added rather than inferred.

Confirming rather than assuming

Where a follow-up is genuinely ambiguous, a short clarifying question costs one turn and prevents an answer built on the wrong assumption.

This is preferable to both guessing and falling back, and customers respond to it readily because the question is specific.