Measure AI Visibility: Method, Metrics and Limits

Blog · Measure AI Visibility

To measure AI visibility, you put real customer questions to several AI systems in a structured way, record every answer and check whether your brand appears, where it appears and how accurately it is described. One question to ChatGPT is not enough, because answers vary from run to run and from system to system. A measurement becomes reliable only with a fixed question set, several systems, repeat runs and clear metrics such as mention rate, position in the recommendation list, factual accuracy and the sources the AI cites. That is exactly how our AI visibility analysis works.

In brief

We measure with a fixed set of realistic questions that never contain your brand name.
Several AI systems and several runs turn single answers into a statement with a clear direction.
The core metrics are mention rate, position, sentiment, factual accuracy and cited sources.
Every measurement is a sample, and an API query is not the same as the chat in a browser.

Published 25 Sep 2026Nikolai Schöbel and Jeremias Burger9 min read

Nikolai Schöbel and Jeremias Burger

Co-founders of Scalableloops GmbH. Nikolai Schöbel leads online marketing and AI strategy, Jeremias Burger the AI architecture. Both build AI systems and train teams on them in their own agency work.

On this page

Why AI visibility is missing from your existing reports
Which prompts belong in an AI visibility analysis?
Which AI systems should you compare?
Which metrics show how visible you are in AI answers?
Who are your competitors in AI answers?
How our AI visibility analysis works
How often should you ask the same question?
Where does every measurement reach its limits?
Which AI visibility tracking tools are available?
What you take away from the measurement
Frequently asked questions
How to start measuring your AI visibility
Where the information on this page comes from

Starting point

Why AI visibility is missing from your existing reports

When someone asks ChatGPT, Gemini or Perplexity for a supplier today, they get a finished answer with a handful of names. None of your classic analytics tools tells you whether your name is among them. Rankings measure where a page sits in search results, traffic measures visits. A recommendation in a chat that later leads to a direct phone call leaves no trace there.

Google does not separate this data either. According to Google's own documentation, sites that appear in AI features such as AI Overviews and AI Mode are counted in Search Console's Performance report under the "Web" search type. There is no dedicated report for these appearances, as Search Engine Journal also points out. If you want to know how your brand performs in AI answers, you have to collect the answers yourself.

This is where our GEO work begins: we measure first, before changing anything. For the terms behind it, read our article GEO, AEO and LLMO explained.

Question set

Which prompts belong in an AI visibility analysis?

A measurement is only as good as its questions. A chat prompt looks different from a Google search: people write full sentences, add budget, region or requirements and ask in the first person. So we do not work with keywords but with questions that sound the way your customers actually ask them. We collect them from at least two sources, for example conversations with your sales team, real customer enquiries and the questions Google itself suggests for a topic.

One rule has no exceptions: your brand name is never in the prompt. We want to know whether the AI names you on its own. A question that already contains your name measures something else, namely what the AI knows about you.

We group the questions by focus areas, the product or service lines where you want to be found. A machine builder has different focus areas than a hotel or an online shop. Per focus area we spread six to eight questions across four prompt types, so that different paths to a buying decision are covered.

Prompt type	Pattern	What it shows
Recommendation	Which provider is right for this problem?	Visibility in general buying advice
Problem	I have this symptom, who can fix it?	Visibility in the language of non-experts
Scenario	As a buyer at a 50-person company I need …	Visibility with decision makers in a specific role
Standard or spec	Which product meets this standard or value?	Visibility in technically demanding questions

Systems

Which AI systems should you compare?

Every AI system answers differently, because each relies on different data, models and search functions. A brand can be recommended regularly in one system and be absent from the next. That is why we ask every question in several systems and evaluate each one separately.

Our standard measurement covers five endpoints: ChatGPT, Claude, Gemini, Perplexity and the AI Overviews in Google Search. Google appears twice on purpose. AI Overviews show what people see in search, while the Gemini model shows how the language model itself treats a topic. We never merge the two into one number. Assistants without a public API, such as Microsoft Copilot, can be checked manually on request, and the report states clearly what was not measured.

How to become more visible in individual systems is covered in our articles on ChatGPT SEO and Google AI Overviews.

5

endpoints in our standard measurement: ChatGPT, Claude, Gemini, Perplexity and Google AI Overviews, each evaluated separately.

Scalableloops, own method

Metrics

Which metrics show how visible you are in AI answers?

For every single answer we record the same fields: whether your brand is named, whether it is backed by a source, where it sits in a list, which products are mentioned, in what tone and how accurate the statements are. These fields produce the metrics, which we read along three questions: Are you noticed? What is said about you? Where do you stand compared with others?

Metric	Question behind it	How we determine it
Mention rate (share of mention)	How often does your brand appear?	Share of answers that include your brand, per focus area and per system
Mention or citation	Are you just named or cited as a source?	Ratio of plain mentions to mentions with a link or source
Position	Are you listed first or last?	Rank of your brand in recommendation lists, as mean and median
Sentiment	How is your brand described?	Each mention rated positive, neutral or negative
Factual accuracy	Is what the AI says about you correct?	Every statement checked against your documents: correct, shortened, wrong or invented
Cited sources	Where does the AI get its knowledge?	List of websites and platforms cited in the answers

Competition

Who are your competitors in AI answers?

In a classic analysis you define competitors in advance. For AI visibility we reverse this: the comparison set emerges from the answers themselves. Which brands the AI names for your topics is a result of the measurement, and often a surprise.

We hold the names from the answers against the list you would have expected. Both kinds of deviation are useful. A competitor you consider strong is missing from the answers. Or a provider you hardly noticed is recommended again and again. In the second case, look at the sources cited for that provider, because that is usually where the explanation lies.

If a competitor is cited with a source while your brand is only mentioned in passing, you usually lack a credible third-party source such as a trade article or an industry directory.

Process

How our AI visibility analysis works

To make a measurement repeatable in three or six months, we fix the process and document every setting: date, model version per system, language and the exact wording of every question. Each question runs in a fresh conversation without history, and we archive the raw answers in full.

01
Collect questions
Step 1
Together with your team and further sources we build a set of real customer questions per focus area.
02
Query
Step 2
Every question runs against all agreed systems, several times and spaced out in the detailed analysis.
03
Evaluate
Step 3
Every answer is recorded in the same schema and checked against your documents.
04
Interpret
Step 4
Metrics per focus area and system, competitor comparison, cited sources.
05
Derive levers
Step 5
Three to five concrete actions, ranked by effort and impact.

Repeat runs

How often should you ask the same question?

AI answers are not fixed. The same question may include your brand today and leave it out tomorrow, so a single answer says little. In our short analysis we ask each question once per system and label the result explicitly as a direction, not as reliable statistics. In the detailed analysis we repeat each question several times at intervals to make the variation itself visible.

When we want to test whether a specific action works, we go one step further. Our GEO audit then uses several runs per question and control brands that have not implemented the same action. Only when the difference to the control group is clear and statistically sound do we call it an effect. Without a control group you cannot tell whether a change comes from your action or from a model update.

In our experience, this is the most common mistake: someone asks once, does not find their name and draws conclusions for their entire marketing.

Limits

Where does every measurement reach its limits?

An AI visibility analysis is always a sample. It shows how your brand performs for a defined set of questions at a given point in time. It does not show how often people actually ask exactly these questions, because the providers do not publish such figures. That is why realistic questions matter so much.

The second limit is the access route. Automated measurements run through the providers' APIs, and these do not necessarily behave like the chat in a browser. With OpenAI, for example, web search has to be added explicitly as a tool in an API request. Without it, the model answers from its trained knowledge, while the chat interface searches the web on its own for many questions. Measuring without web search therefore measures the model's knowledge, not necessarily what your customer sees in the browser. Every report we deliver states which mode was used.

Third, the systems keep changing. A measurement is therefore a dated snapshot, not a permanent state.

Limit	What it means	How we handle it
Sample	Only the questions asked are measured	Realistic question set from several sources, result labelled as a direction
Varying answers	The same question returns different answers	Repeat runs in the detailed analysis, control brands in the GEO audit
API instead of chat	Without web search the model answers from its knowledge	Measurement mode per system named in the report
Systems without an API	Some assistants cannot be queried automatically	Manual check on request, otherwise the gap is stated explicitly
Point in time	Models and sources change	Date, model version and questions documented, repeat measurement possible

Tools

Which AI visibility tracking tools are available?

Besides your own measurements, there are now tools that track prompts over time. According to its own description, SISTRIX measures where and how a brand appears in AI answers, including ChatGPT, Perplexity and Google AI Overviews, together with cited sources and competitors. The Semrush AI Visibility Toolkit tracks visibility for selected prompts in, among others, ChatGPT and Google's AI features, and reports mentions, sentiment and share of voice. Both descriptions come from the vendors and are confirmed by independent overviews.

Such tools are strong at ongoing monitoring over weeks and months. They do not replace the work on the question set or the check whether statements about your brand are correct. So we use both: our own measurements with your questions and third-party data as a cross-check. What comes next, from llms.txt to work on third-party sources, is covered in our article on AI SEO.

Outcome

What you take away from the measurement

The result is not a pile of numbers but a clear overview: how present you are in each focus area, which competitors are named ahead of you, where the AI says something wrong about you and which sources it relies on. From this we derive three to five actions, ranked by effort and impact. Typical ones are maintaining third-party sources, content with precise and verifiable facts, clean structured data on your own website and publications in trade media.

Because questions, systems and settings are documented, the same measurement can be repeated after implementation. You see whether your mention rate, your position and factual accuracy have actually changed. We cannot promise a specific result, because the answers come from systems nobody but the providers controls. What we do promise is a clean, traceable measurement.

Frequently asked questions

Can I measure my AI visibility myself?

Yes, on a small scale. Ask a few realistic customer questions without your brand name in several AI systems, each in a new chat, and note mention, position and accuracy. A reliable result needs a fixed question set, several systems, repeat runs and consistent evaluation.

Why is one question to ChatGPT not enough?

AI answers vary. The same question can name different providers in a new run. A single answer is a matter of chance; only many answers show a direction.

What is the mention rate?

The mention rate, also called share of mention, is the share of AI answers that include your brand. We report it separately per focus area and per AI system.

Does Google Search Console show my visibility in AI Overviews?

Not separately. According to Google, appearances in AI Overviews and AI Mode are counted in the Performance report under the Web search type, without a dedicated breakdown.

Does an API query measure the same thing as the chat in a browser?

Not necessarily. With OpenAI, web search has to be added explicitly as a tool in an API request. That is why every report states which mode was used.

How often should the measurement be repeated?

A repeat makes sense after actions have been implemented and after major model changes, in many cases every three to six months. What matters is that questions and settings stay the same, otherwise results are not comparable.

Next steps

How to start measuring your AI visibility

01
Define focus areas
Pick the two or three service lines where you want to appear in AI answers.
02
Collect customer questions
Ask sales and customer service how customers phrase their needs and write these questions down word for word.
03
Run a first sample
Ask some of these questions without your name in several systems and note who gets named.
04
Set up a proper measurement
For a reliable competitor comparison and a repeatable measurement, get in touch. If your team should run the measurement itself, it learns how in our AI training.

Whether an AI recommends your brand cannot be guessed, but it can be measured properly. Knowing your own numbers lets you invest exactly where the gap really is.