Measuring AI Search When Most Of It Can't Be Tracked

The visibility dashboards are the new vanity metric. Here's how I'd actually tell whether AI search is doing anything for your business.

AI visibility isn't a fixed rank. Ask the same thing twice in two sessions and you can get two different shortlists. It moves with the model, the wording, and whatever the platform remembers about the person asking. A tidy share-of-voice figure flattens all of that and hands it back to you as if it were a stable position.

It isn't one channel either. In our study of 115 Australian businesses, ChatGPT drove about 90% of identifiable AI visits and converted the worst. Perplexity and Gemini sent fewer people who bought three to six times more often. SE Ranking's cross-platform data shows the same spread. Average those platforms into one score and you've buried the only useful part of it.

It's uneven across businesses too. The brands that get surfaced already have a recognisable name, consistent details across the web, real reviews, and third-party coverage. That's years of groundwork, not a setting you switch on.

Here's how I search now, and probably you. Ask ChatGPT a question, three names come back, Google the one that stuck, click through, buy. The AI made the decision. GA4 files it under direct or branded organic. The platform that did the work gets none of the credit.

That's not a hunch anymore. Similarweb's clickstream study found brands ChatGPT recommended were 2.5 times more likely to get a site visit within a week, and that 55.9% of it arrived through branded search rather than a visible AI referral. The AI persuaded, Google took the credit. So better tracking won't rescue you here. Half the effect was never going to land in an AI column.

You measure it like a leading indicator, not a result. A few things that hold up.

• Watch the signals together. When AI visibility lifts, branded search, direct traffic and assisted conversions should lift in the same window. Three of four moving is a pattern worth trusting. Visibility soaring while branded search flatlines is not.
• Make branded demand the scoreboard. Branded search and direct traffic are the cleanest proof you have that AI is putting you in front of people. They already sit in GA4 and Search Console, so no new platform required. It's the spine of how we build AI search reporting for clients.
• Track the prompts your buyers actually run. Ten commercially relevant prompts beat 500 vanity ones. A dashboard shows you the easiest thing to measure, which is rarely the thing that matters.
• Split the platforms out. ChatGPT, Perplexity and Gemini behave differently enough that an average across them tells you close to nothing.
• Know your own noise. If you run a tracker, work out how much of your AI interest is the tool itself. Point it at a quiet set of URLs and watch the floor before you trust the trend.
• For service businesses, count inclusion before clicks. If your name shows up in answers more often while referrals stay thin, that's your early signal. It's word of mouth, basically. Mentioned first, visited later.
• Build the foundations that get you cited. Consistent entity information, real reviews, third-party coverage, clean structured content. It's the same foundational SEO work you were meant to be doing, and an AI visibility audit will show you where you stand today.

Most of what's worth having is first-party and free. The honest version of each:

• GA4. Since May it has a native AI Assistant channel, so clicks from ChatGPT, Gemini and Claude show up on their own instead of buried in Referral. Two catches. It only counts forward from the day it switched on, and it misses the big stuff. Perplexity still lands in Referral, referrer-less visits land in Direct, and Google's own AI Overviews get filed as organic. Read it as a floor.
• Search Console's AI report. Google's new generative AI performance report shows how often your pages turn up in AI Overviews and AI Mode. It's impressions only though, no clicks and no queries, so it tells you that you're visible, not whether it did anything.
• Bing Webmaster Tools. Underused, and for once the most generous of the lot. Its AI Performance report shows where Copilot and Bing actually cite you, plus the grounding queries behind it, for free. The catch is scope. It's the Microsoft side of the web, not ChatGPT or Google.
• Shopify's agentic reporting. If you're on Shopify, agentic orders land in the admin tagged with the AI channel that drove them, about the closest thing to clean first-party AI sales attribution going. It only sees sales that actually transacted through the channel, so the ChatGPT-then-Google path still slips past it.
• Ahrefs or Semrush. Both now track brand mentions and share of voice across the AI platforms the first-party tools ignore. Useful for competitive context, whether you're cited more or less than your rivals, as long as you remember it runs on sampled prompts rather than real users.
• The hundred prompt trackers. Profound, Peec, Otterly and the rest all do a version of the same thing, and some charge like they don't. Pick one if you want the competitive view, watch for the tool measuring its own scans, and don't pay enterprise money for a prettier citation count.

Every one of these measures arrival or a citation, not influence. Tying them back to money is the job the branded-search read above is doing.

The brands that win the next few years won't be the ones with the biggest visibility number. We polished a scoreboard exactly like this with rankings for a decade before we admitted that finishing first isn't the same as getting paid.

The winners will be the ones who work out which slice of their AI visibility actually moves money, and have the discipline to let the rest go.