Confidence thresholds determine how the platform categorizes each AI response into high, medium, or low confidence bands. These bands drive escalation rules, dashboard metrics, and the confidence labels shown in conversation logs — tuning them is one of the fastest ways to change how the AI assistant behaves without touching any content. The page also shows a Minimum Reranker Threshold slider; the value you set there now persists when you save it, but the orchestrator does not yet apply it — see Minimum Reranker Threshold below.
The Confidence Thresholds page combines the band slider with the Minimum Reranker Threshold control.
Viewing and saving confidence thresholds requires the settings_confidence permission, which is held by the Platform Admin role (assigned to users or groups under Settings → Access). Members do not see the Confidence Thresholds item in the Settings navigation, cannot open the route, and the /api/v1/tenant/confidence-settings API returns 403 for both reads and saves.
Confidence Bands
The upper panel displays a dual-handle slider. Its rail is labelled 0% and 100%, but the two handles travel only between 1% and 99%, and they can never meet: the API rejects 0 and 1 as threshold values and additionally requires Medium to stay strictly below High, so the slider clamps to the reachable window (step 1%, minimum one step between the handles) rather than producing a pair the save would reject.
Drag the handles to set two boundaries that split every scored response into three bands:
- High confidence — responses scoring at or above the upper threshold (default: 90% and up)
- Medium confidence — responses scoring at or above the lower threshold but below the upper one (default: 60% up to, but not including, 90%)
- Low confidence — responses scoring below the lower threshold (default: below 60%)
A score that lands exactly on a boundary is classified into the higher band.
The slider uses a color gradient — green for high, yellow for medium, and red for low — so you can see the relative size of each band at a glance.
Previewing the Impact
When you adjust either threshold, a reclassification preview appears below the slider. The preview analyzes the last 30 days of conversation data and shows:
- Current vs. proposed counts — how many conversations fall into each band under the current and proposed thresholds
- Reclassification impact — conversations that would move to or from the Medium band (High → Medium, Medium → Low, Low → Medium, Medium → High)
This helps you understand the effect of a change before you save it.
The reclassification list is not a complete account of the change. It only reports moves that touch the Medium band — a conversation that jumps straight from High to Low (or Low to High) is discarded before it reaches the preview. If your proposed bands cause only such jumps, the reclassification section does not render at all, even though the Current vs. Proposed band counts visibly shift. Trust the Current vs. Proposed counts, not the reclassification total, as the measure of how much a change moves.
The preview uses 30 days of historical data. If your tenant is new or has low conversation volume, the preview may show limited results — make small threshold changes and watch the dashboard for a day or two before tightening them further.
Minimum Reranker Threshold
The second panel on the page holds a single slider labelled Minimum Reranker Threshold, shown at 0.30. The relevance gate it describes is real, and the value you set here is now saved — but the orchestrator does not yet consult it, so changing it does not change how the gate behaves.
Your saved value persists, but it does not take effect yet. Saving a reranker threshold now writes it to the tenant's settings and reads it back, so the slider keeps the value you chose across a reload instead of snapping back to 0.30. What it does not do is change retrieval: the orchestrator still evaluates the relevance gate at the default 0.3 for every tenant, regardless of the value on this slider. Enforcing a tenant-specific value is deliberately deferred — raising the gate today would flood the escalation queue — so treat the effective gate as 0.3 until this page says otherwise.
There is currently no way to make the assistant decline more or less often by moving this slider. To change how often it declines, improve knowledge base coverage for the questions it is failing, and add an Escalation Rule with the Create Ticket action on the low-confidence band so those questions reach a human. The band sliders in the panel above are fully wired and do take effect.
The gate itself runs after knowledge base retrieval and before answer generation — earlier in the pipeline than the confidence bands, which classify answers that have already been generated. The orchestrator compares the highest reranker score from the retrieval step against 0.3 (the default — the saved slider value is not yet consulted):
- If the best match scores
0.3or higher — the orchestrator generates an answer and scores it for confidence as usual - If every result scores below
0.3— the orchestrator skips generation entirely and returns a fixed low-confidence reply ("I couldn't find sufficient information to answer your question…") instead of a generated answer
The same floor is applied on the streaming and non-streaming paths. A curated answer served straight from the knowledge base does not pass through it.
Saving Threshold Changes
Adjust the Thresholds
Drag the band slider to change the High and Medium boundaries. The Minimum Reranker Threshold slider now saves and keeps its value across a reload, but the orchestrator does not yet apply it — moving it has no effect on retrieval today.
Review the Preview
Check the impact preview under the sliders to confirm the effect of your proposed bands. Read the Current vs. Proposed band counts — the reclassification list below them omits any conversation that moves directly between High and Low.
Save Changes
Click Save Changes at the bottom of the page. The button is disabled until you make a change. To undo your edits without saving, click Discard.
A confidence band is not stored on a conversation — it is derived from the stored score against the current thresholds every time it is read. Saving new thresholds therefore re-bands your history immediately: dashboard band counts and success rate for periods that have already elapsed will shift, which is exactly what the impact preview is showing you. The stored confidence scores themselves are never rewritten, and escalation rules that already ran against past conversations are not re-evaluated.
Field Reference
| Field | Description | Range | Default |
|---|---|---|---|
| High Threshold | Minimum score for the High confidence band | 2–99% — 0% and 100% are unreachable, and the High handle cannot sit on or below the Medium handle | 90% |
| Medium Threshold | Minimum score for the Medium confidence band | 1–98% — always at least one percentage point below High | 60% |
| Minimum Reranker Threshold | Minimum reranker score required before the orchestrator generates an answer | Slider offers 0.0–1.0 (step 0.05); the value is saved and persists across reloads, but the orchestrator does not yet apply it | Effective gate is 0.3 for every tenant — the saved value is not yet enforced |
Related Pages
- Escalation Rules — define automated actions based on confidence bands
- Model Routing — configure the confidence assessment model used to score responses
- Dashboard — monitor confidence distributions and success rate trends
- Conversation Logs — review audit records with confidence scores