there’s a <1 in 100 chance that ChatGPT or Google’s AI, if asked 100X, will give you the same list of brands in any two responses
it’s more like 1 in 1,000 runs before you’d see two lists in the same order
any tool that gives a “ranking position in AI” is full of baloneyAnd a piece of advice to buyers that we are going to take literally:
stop throwing money at AI tracking products that don’t provide stats-backed, publicly-reviewable research. Before you spend a dime tracking AI visibility, make sure your provider answers the questions we’ve surfaced here and shows their math.This page is rekkal showing its math.
What the research means for a tracking tool
If any single answer is close to a coin flip, then:- One sample is not a measurement. A number from one run of one prompt tells you what the engine said once. It does not tell you what the engine says.
- A day-over-day chart is a noise chart. Most of the movement in a daily line is resampling, not the world changing.
- A rank position is not a fact. Order is less stable than membership, so a “position in AI” figure is the least trustworthy thing such a tool can print.
I now believe visibility % across dozens to hundreds of prompts run multiple times is a reasonable metric
What rekkal does about it
Repeat the same prompts on a fixed schedule
Count answers, not name-drops
Put an exact confidence interval on every estimate
Only claim a change when the intervals are disjoint
Give average position no direction at all
Render absence as absence
The one place rekkal writes a zero
Honesty about a rule includes its exception. If a week ran, and a brand in your tracked set was named in none of that week’s answers, that brand’s figure for that week is 0% — not “missing”. It was measured against a real denominator and went unmentioned, which is an observation. Everything else absent stays absent. A week with no runs contributes nothing at all: no zero, no point, no interpolation.What rekkal is measuring, precisely
- The consumer interfaces, signed out — the ChatGPT web app, the AI Overview on a signed-out Google results page, and the Copilot web app. This is what a person without an account is shown. Which of them your plan covers is on Plans and limits.
- No account, anywhere. rekkal holds no login on any of these platforms, so no answer it reads is personalised to a profile or shaped by a chat history — and there is no credential for a provider to suspend. Two of the three are read in a real browser session driven by rekkal; the Google surface is retrieved through SerpApi, a vendor that sells search-results retrieval and carries the legal indemnity for it.
- One fresh session per answer. Cookies, storage and the exit address are all rotated. That is a statistical requirement rather than a nicety: answers drawn through one warm session are correlated, and correlated draws make a confidence interval narrower than the data supports — which is the exact failure the rest of this page exists to avoid.
- No challenge is ever solved or bypassed. Where a bot-defence check stands between rekkal and an answer, that run is recorded as a failure. It is not retried until it works, and the week’s sample is smaller and says so.
- One market and language per workspace, derived from your domain at setup. rekkal does not currently run the same prompt across several countries or languages in parallel.
Known limits of this approach
We would rather you read these here than discover them later.Daily cadence buys sample, not certainty
Daily cadence buys sample, not certainty
Sample size is bounded by your prompt allowance
Sample size is bounded by your prompt allowance
n is prompts × engines × days. On a smaller plan, intervals stay wide for longer,
and rekkal will keep telling you so rather than pretending otherwise.Which model answered is not knowable, and rekkal does not guess
Which model answered is not knowable, and rekkal does not guess
Whether the platform searched is observed, not requested
Whether the platform searched is observed, not requested
Market and language are where rekkal asked from, not what it asked for
Market and language are where rekkal asked from, not what it asked for
An interface can change, and then a week is missing rather than wrong
An interface can change, and then a week is missing rather than wrong
Mentioned is not yet distinguished from recommended
Mentioned is not yet distinguished from recommended
Only the visibility figure opens onto its answers
Only the visibility figure opens onto its answers