How to Read Your Chatbot's Analytics Dashboard (What to Check First)
A priority order for reading chatbot analytics — the one metric that tells you what to fix, and four more that only make sense once you check it.
Open a chatbot analytics dashboard for the first time and every number looks like it matters equally. It doesn't.
If you only check one thing, check unanswered questions — the list of things visitors asked that your chatbot couldn't confidently answer. It's the only metric on the dashboard that tells you exactly what to fix, not just how you're doing.
Here's the order to check the rest in, and what each one is actually telling you.
The five numbers on a chatbot dashboard
Most chatbot analytics dashboards — EmbedMyBot's included — boil down to five things:
- Unanswered questions (knowledge gaps). Questions the chatbot couldn't confidently answer from your trained content, grouped and ranked by how often they come up.
- Deflection rate. The share of conversations resolved without a human stepping in.
- CSAT. The rating a visitor leaves on an answer, right inside the chat.
- Top questions. What visitors ask most, grouped by topic.
- Conversation volume. How many conversations happened, by day, week, or month.
Two more show up on most dashboards and are worth knowing, but they're context rather than a to-do list: response time (how fast the bot replies) and busiest hours (when conversations happen). Useful for staffing and expectations. They won't tell you what to fix.
The mistake most guides to this make is treating all seven as one flat list, reviewed in whatever order they're printed on the page. They're not equally useful, and they're not independent of each other. Here's the order that actually matters.
Check unanswered questions first
Every other number on the dashboard describes how the chatbot performed. This one describes what it's missing — and that's the only thing you can act on directly.
When your chatbot can't find a confident answer in whatever it's trained on — your docs, your website, your FAQ — a well-built one says so instead of guessing. That "I don't know" moment is the whole point: it's a logged, specific, fixable gap instead of a customer quietly getting a wrong or made-up answer and not telling anyone.
The fix loop is the same every time:
- 01Open the gap list and sort by frequency, not recency. A question ten people asked this week matters more than one asked once yesterday.
- 02Check whether the answer actually exists somewhere in what you've trained the chatbot on. Sometimes it does, and it's just phrased differently, buried in a PDF, or split across two documents the retrieval didn't connect.
- 03If the answer doesn't exist yet, write it — a paragraph on an existing page, a new FAQ entry, a short doc. It doesn't need to be long. It needs to directly answer the exact question that was asked.
- 04Re-crawl or re-upload, then ask the chatbot the original question yourself to confirm it's fixed before you move to the next gap.
This is the actual difference between a chatbot that gets more useful over time and one that plateaus after setup week. The gap list is a running record of exactly what your content doesn't cover yet — you don't have to guess, and you don't have to comb through transcripts to find the pattern.
Not every gap is worth fixing. A one-off question from a single visitor about something oddly specific to their situation isn't a content gap — it's an outlier. Grouping by frequency is what separates "add this" from "ignore this."
Then check deflection rate and CSAT — together
These two get compared separately in most write-ups, which misses the actual point: they're only meaningful read side by side.
A rising deflection rate on its own sounds like good news — more conversations resolved without a human. But a chatbot trained on your own content can resolve a conversation confidently and still be wrong, especially on questions that are close to something it knows but not quite. That's what CSAT is for.
If deflection rate climbs while CSAT holds steady or improves, the bot is genuinely getting better. If deflection rate climbs while CSAT drops, it's resolving more conversations by guessing — and visitors are quietly telling you that in the ratings.
Read as a pair, they answer a question neither one answers alone: is this chatbot handling more on its own because it's actually good, or because it's confidently wrong more often?
| Deflection rate | CSAT | What it means |
|---|---|---|
| Rising | Steady or rising | Genuinely getting better — keep doing what you're doing |
| Rising | Falling | Resolving more by guessing — check recent "answered" conversations, not just the gap list |
| Falling | Steady or rising | Probably a mix of harder or more varied questions coming in, not a quality problem |
| Falling | Falling | Something changed in your source content, or a recent edit introduced an error — check what was updated last |
Top questions, volume, and busiest hours are context, not a to-do list
These three are worth a glance, not a deep review. Top questions tells you what visitors care about — useful for deciding what to feature on the page the chatbot sits on, less useful for finding what's broken (that's what the gap list is for). Conversation volume and busiest hours are mostly useful for noticing something unusual: a spike after a marketing push, a dead period worth investigating, load that tells you when a support handoff needs to be staffed.
None of them tell you to change anything by themselves. Treat them as the backdrop the first three numbers sit in front of.
A five-minute weekly review
You don't need a dashboard meeting for this. A workable weekly routine:
- 01Open the gap list. Fix the top one or two if they're a genuine pattern (multiple people, same underlying question).
- 02Glance at deflection rate and CSAT together. If they're moving in opposite directions, spend five minutes reading a handful of recent "resolved" conversations before assuming it's fine.
- 03Skim top questions for anything new that wasn't showing up last week — an early signal before it becomes a bigger gap.
- 04Everything else — volume, busiest hours, response time — only needs a look if something else prompted the check (a launch, a complaint, a slow week).
That's it. The whole point of ranking gaps by frequency and pairing deflection rate with CSAT is that you're not scrolling transcripts looking for patterns by hand — the dashboard has already done the grouping.
Getting chatbot data into your existing analytics
If your site already runs Google Analytics 4 or Google Tag Manager, you don't have to treat chatbot data as a separate silo you check in a different tab. A properly built widget can auto-forward chat events — opened, message sent, lead captured, CSAT rated, handed off to a human — straight into your existing dataLayer, tagged with which chatbot they came from.
That means chatbot activity shows up next to the rest of your site's behavior instead of living in a dashboard nobody remembers to open. The exact event list and setup steps are in the analytics documentation.
Frequently Asked Questions
What's a good deflection rate for a chatbot?
There isn't a universal number worth chasing — it depends heavily on how narrow or broad the questions you get are. What matters more than the number itself is the trend, and specifically whether it moves together with CSAT. A high deflection rate paired with falling satisfaction is worse than a moderate one that's climbing alongside good ratings.
What counts as an "unanswered question" or knowledge gap?
It's a question the chatbot couldn't confidently answer from whatever it's trained on — your documents, crawled pages, or pasted content — so it returned an honest "I don't know" instead of guessing. A well-built chatbot logs these automatically and groups similar phrasings together so the same underlying gap doesn't look like ten separate ones.
Do I need a separate analytics tool for my chatbot?
Not if the platform already tracks the five numbers above and lets you forward events to GA4 or GTM. A separate tool adds value mainly for teams running conversational AI at a scale where they need cross-channel attribution modeling — most small business chatbot deployments don't need that overhead.
How often should I check chatbot analytics?
Weekly is enough for most sites. The gap list and the deflection-rate/CSAT pairing are the two things worth a standing five-minute check; everything else only needs attention when something prompts it, like a traffic spike or a customer complaint.
Is chatbot analytics available on a free plan?
On EmbedMyBot's free plan, yes — the full dashboard (volume, top questions, deflection rate, CSAT, and knowledge gaps) is available from your first conversation, with no separate reporting tier to upgrade into.
What's the difference between deflection rate and CSAT?
Deflection rate measures whether a human had to step in. CSAT measures whether the visitor was actually satisfied with what happened. A chatbot can resolve a conversation with no human involved and still leave the visitor unhappy — which is exactly why the two need to be read together, not separately.
Ready to see your own gap list instead of guessing what to write next? Start free on EmbedMyBot and the dashboard starts tracking from your first conversation — no credit card required.
