Tracking AI Overviews: Why You Have to Measure It Apart From Chatbots

Chatbot citations are up, but you're nowhere on Google

About two months after one team started with GEO, their ChatGPT citations climbed noticeably. The brand began showing up in answers for half the questions they were tracking, and the report graph pointed up and to the right. Then sales surfaced a different story. A prospect had searched the category on Google, and the summary box at the very top held only two competitors. Same company, same quarter, opposite results.

This mismatch wasn't caused by measuring wrong. It happened because two different surfaces were lumped together as one. Chatbots like ChatGPT and Google AI Overview both look like AI-synthesized answers. But the trigger that produces the answer, the shape of the incoming question, and the logic for choosing sources are all different, so what works on one side doesn't automatically carry to the other. Which means you can't average them in a single column either.

The overview posts you usually see stop at "go ask ChatGPT, Perplexity, and AI Overview directly." Let's take a step further than that. This piece pulls out AI Overview alone. We'll walk through why it's a different surface from chatbots, how to track it separately, and what that tracking does and doesn't actually measure.

AI Overview is a search surface, not a chatbot

First, one misconception to clear up. Google AI Overview isn't "Google's chatbot." It's a summary answer that slots in at the very top of the search results page. The user didn't open a chatbot app and start a conversation. They typed something into the search box like they always do, and a tidied-up answer happens to sit on top first.

This seemingly trivial difference changes the premise of measurement, because chatbots and search-style answer surfaces start from entirely different contexts.

The last row of the table matters most. AI Overview shares the same screen as the traditional search results sitting beneath it. So it can't be unrelated to search ranking. The better a page is indexed and trusted in search, the higher its odds of becoming material for the summary. So far, chatbot citations come out of a mix of model training and real-time retrieval augmentation. AI Overview, by contrast, is best seen as a layer laid on top of search.

If chasing chatbot citations is about persuading the model, chasing AI Overview is closer to getting a page trusted in search picked for the summary.

How AI Overview works on search-style questions

The key to understanding AI Overview is how the user asks, because people don't use the same words in a chatbot and a search box.

Into a chatbot, you throw long, context-loaded sentences. Something like "we're a 20-person startup, recommend a tax tool that handles both invoices and payroll in one place, and it'd be nice if the pricing isn't a burden." You string conditions together. Into a search box, you type clipped phrases like "tax SaaS recommendations," "small business tax software comparison," or "free invoicing."

Given a short query like that, AI Overview builds the answer roughly along these lines.

1. Decide: First it judges whether this query is the kind worth showing a summary answer for. Queries with something to synthesize, such as definitions, comparisons, and how-tos, get a summary, while simple browsing or transactional queries sometimes don't. So even for the same topic, an overview may appear or drop out depending on how you search.
2. Gather: It pulls relevant pages from the search index, and traditional search trust influences this step.
3. Synthesize: It picks the answering fragments from the pulled pages, bundles them into a single summary, and attaches source link cards beside it.

The crux is in the first step. A chatbot will produce an answer no matter what you ask. AI Overview, on the other hand, first decides whether to appear at all. That's why tracking results split three ways: the overview appeared and we were cited, the overview appeared but we were left out, or no overview appeared at all. That last state doesn't exist in chatbot tracking. "Not appearing in the answer" on a chatbot and "no overview showing up at all" on AI Overview are entirely different signals, so mixing them means you misread both.

Why you have to view it separately from chatbot citations

Viewing the two surfaces separately isn't about tidiness. It's because combining them throws off your decisions. Three reasons, specifically.

First, one side's wins mask the other

The opening example is exactly this case. When your chatbot citation rate is rising while AI Overview presence sits at the bottom, averaging the two and reporting "AI citations XX%" buries both the wins you earned on chatbots and the risk of being empty on AI Overview at once. Split by surface and you get the next move: "chatbots are handled, now let's go after the search surface."

Second, the work to win a citation is different

To get cited in a chatbot, you weight building verifiable answer fragments the model can lift out as a single clean sentence. AI Overview adds to that: you have to target the search-style queries that trigger a summary in the first place, plus the work of becoming a page trusted within the search index. The two jobs overlap a lot, but they aren't the same. So if you don't know which surface is empty, you can't know which job to do.

Third, they play different roles in the user journey

The same person meets the two surfaces at different moments. When vaguely exploring a category, they ask a chatbot at length and get candidate recommendations; when checking a specific brand or fact, they run a short Google search and read the AI Overview summary. If you're only in one, you vanish from half the journey, so "where are we empty" becomes a question that has to be answered surface by surface.

How to track AI Overview

AI Overview tracking diverges from chatbots right from the start. With chatbots, you can just ask the questions you'd normally throw at them. But for AI Overview you have to convert those questions into search-style queries. There's one extra step of rewriting them into the short phrases people actually type into a search box. Here's the workflow.

1. Convert to search-style queries. For each tracking topic, build the short query a user would actually enter in the search box, pulling forms likely to trigger a summary, such as comparison, definition, and how-to, using "what would someone looking for recommendations in our category type" as the yardstick.
2. Record whether an overview appeared first. Before judging citation, log whether the overview itself even showed up. A query with no overview isn't a loss but a separate status, and over time a summary may start appearing for that same query.
3. Check the citations inside the overview. If an overview appeared, look at whether your brand is mentioned in the body, whether your page is in the link cards beside it, and which competitors made it in.
4. Store it in a separate column from chatbots. This is the most important operating principle. Even for the same citation, you have to tag which surface it came from and aggregate separately, so you can read each surface's citation rate and one side doesn't mask the other.
5. Re-measure on a regular cadence. AI Overview summaries change with the state of the search index, so an overview that once appeared can vanish and the citation mix can shift. Read it as a trend, not a single snapshot.

Storing it split by surface is also the practical crux here. If your citation records don't mark whether they came from a chatbot response or a search-style overview, the numbers from the two surfaces get mixed into one bucket. And then the per-surface citation rate metric never even gets built.

The limits this tracking has to own up to

There's a part to call out honestly. AI Overview tracking is useful, but you shouldn't overstate what it measures. GEO is still an early field and the search-side logic isn't public, so you have to carry these limits clearly.

• What it measures is "did an overview appear for the queries we created," not "did the customer actually search that way." The fact that an overview appeared for the search-style query we built and we got cited in it is a signal that we can show up for someone searching that query. But what the real search volume is, and how well that query represents customers' actual searches, are separate questions. So choosing tracking queries well determines the validity of the results.
• Results wobble with personalization, location, and device. Search is context-sensitive, so whether an overview appears and how it's composed can change with who searched, from where, and on what device. The screen you saw once isn't the same screen for everyone.
• There's high volatility in "appeared / didn't appear." Which queries AI Overview shows a summary for keeps getting tuned, so an overview that appeared yesterday may not appear today. So don't over-trust the conclusion of a one-off measurement. Read it as a trend by repeating the same query.
• A citation on screen isn't traffic. Being cited in a summary doesn't guarantee a click follows, so exposure and inbound should be read as different metrics. AI Overview tracking only measures whether you were exposed at the answer stage; it doesn't directly measure inbound performance.

These limits don't mean don't track. They mean know exactly what you're measuring and use it accordingly. For the bounded question of "how often does an AI Overview summary appear across the search-style queries we created, and how much do we get cited in it versus competitors," surface-separated tracking gives a far clearer answer than guessing by gut.

The takeaway: don't average, view by surface

It all comes down to one point. AI Overview is a different surface from chatbots, so you have to measure it separately to keep your decisions from going sideways. The moment you roll chatbot citations and AI Overview presence into one column and average them, you hide both the wins on the strong side and the risk on the empty side.

In practical order, it goes like this. Convert your tracking topic into search-style queries, record whether an overview appeared first (separately from chatbots), view the citations inside the overview alongside competitors, then re-measure on a cadence. As you read the results, never forget the caveat: "this is overview exposure on the queries we created, not an actual citation from a customer's real search."

Tracking the two surfaces separately and not combining them is something you can do by hand. But converting queries into search style, aggregating by surface, and re-running on a cadence gets tedious fast. NUDGEO helps you start from one place: seeing, surface by surface, how your chatbot citations and Google AI Overview presence are showing up right now.