> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tryrekkal.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Methodology

> The independent research on AI answer inconsistency, and exactly what rekkal does about it — sampling, statistics, and the claims it refuses to make.

In January 2026, Rand Fishkin published the only independent study of this question we can
find: **2,961 human-run prompt executions** across ChatGPT, Claude and Google AI, on
[SparkToro](https://sparktoro.com/blog/new-research-ais-are-highly-inconsistent-when-recommending-brands-or-products-marketers-should-take-care-when-tracking-ai-visibility/).

Its findings are inconvenient for this entire product category:

> there's a \<1 in 100 chance that ChatGPT or Google's AI, if asked 100X, will give you
> the same list of brands in any two responses

> it's more like 1 in 1,000 runs before you'd see two lists in the same order

> any tool that gives a "ranking position in AI" is **full of baloney**

And a piece of advice to buyers that we are going to take literally:

> stop throwing money at AI tracking products that don't provide stats-backed,
> publicly-reviewable research. Before you spend a dime tracking AI visibility, make sure
> your provider **answers the questions we've surfaced here and shows their math**.

This page is rekkal showing its math.

## What the research means for a tracking tool

If any single answer is close to a coin flip, then:

1. **One sample is not a measurement.** A number from one run of one prompt tells you what
   the engine said once. It does not tell you what the engine says.
2. **A day-over-day chart is a noise chart.** Most of the movement in a daily line is
   resampling, not the world changing.
3. **A rank position is not a fact.** Order is *less* stable than membership, so a
   "position in AI" figure is the least trustworthy thing such a tool can print.

Fishkin's own conclusion points at the one thing that does work:

> I now believe **visibility % across dozens to hundreds of prompts run multiple times** is
> a reasonable metric

## What rekkal does about it

<Steps>
  <Step title="Repeat the same prompts on a fixed schedule">
    Every prompt is re-run on every covered engine, every week. Your sample grows every
    week whether or not anything changed, and the comparison is always like-for-like:
    the same prompts, the same engines, the same slot length.

    The unit rekkal records — and bills — is one prompt × one engine × one daily slot. That
    is not an implementation detail we happen to mention; it is the number every figure on
    your dashboard is divided by, and it is printed on screen.
  </Step>

  <Step title="Count answers, not name-drops">
    An answer that mentions your brand five times is **one** observation. Counting each
    mention separately would inflate the sample and make every confidence interval narrower
    than the evidence justifies — which is how a tool ends up confidently reporting a
    movement that is not there.
  </Step>

  <Step title="Put an exact confidence interval on every estimate">
    Clopper–Pearson exact binomial intervals, on the headline cards, on every per-engine bar,
    on every row of the competitor ranking, and as a band on the charts. Small sample, wide
    band — visibly.
  </Step>

  <Step title="Only claim a change when the intervals are disjoint">
    A direction and a colour are shown **only** when this period's interval and the previous
    period's interval do not overlap. Everything else reads *No measurable change*, with the
    point movement stated so you can see what was suppressed and why.

    This rule lives in one function, and the components that draw the arrows take their
    colour from its verdict. A component cannot draw an arrow on noise even by mistake.
  </Step>

  <Step title="Give average position no direction at all">
    Because the research is clearest here, and because the arithmetic agrees: rekkal stores
    no variance for a mean, so there is no interval to test with. Both periods are shown
    side by side and the verdict is *Change not testable*. rekkal will not print a "your
    rank improved" claim, on any plan, at any sample size.
  </Step>

  <Step title="Render absence as absence">
    A week that did not run has no point on the chart and nothing is interpolated across it.
    A missing comparison names which half is missing. There is no zero-filling anywhere in
    the read path — with one deliberate exception, below.
  </Step>
</Steps>

## The one place rekkal writes a zero

Honesty about a rule includes its exception.

If a week ran, and a brand in your tracked set was named in none of that week's answers,
that brand's figure for that week is **0%** — not "missing". It was measured against a real
denominator and went unmentioned, which is an observation.

Everything else absent stays absent. A week with no runs contributes nothing at all: no
zero, no point, no interpolation.

## What rekkal is measuring, precisely

* **The consumer interfaces, signed out** — the ChatGPT web app, the AI Overview on a
  signed-out Google results page, and the Copilot web app. This is what a person without an
  account is shown. Which of them your plan covers is on [Plans and limits](/plans).
* **No account, anywhere.** rekkal holds no login on any of these platforms, so no answer it
  reads is personalised to a profile or shaped by a chat history — and there is no
  credential for a provider to suspend. Two of the three are read in a real browser session
  driven by rekkal; the Google surface is retrieved through SerpApi, a vendor that sells
  search-results retrieval and carries the legal indemnity for it.
* **One fresh session per answer.** Cookies, storage and the exit address are all rotated.
  That is a statistical requirement rather than a nicety: answers drawn through one warm
  session are correlated, and correlated draws make a confidence interval *narrower* than
  the data supports — which is the exact failure the rest of this page exists to avoid.
* **No challenge is ever solved or bypassed.** Where a bot-defence check stands between
  rekkal and an answer, that run is recorded as a failure. It is not retried until it works,
  and the week's sample is smaller and says so.
* **One market and language per workspace**, derived from your domain at setup. rekkal does
  not currently run the same prompt across several countries or languages in parallel.

## Known limits of this approach

We would rather you read these here than discover them later.

<AccordionGroup>
  <Accordion title="Daily cadence buys sample, not certainty">
    A daily slot means a genuine change takes days rather than weeks to become statistically
    visible. That is the honest cost of not reporting noise. If you need to know within 24
    hours that something moved, no sampling-based tool can honestly give you that — including
    this one.
  </Accordion>

  <Accordion title="Sample size is bounded by your prompt allowance">
    Your `n` is prompts × engines × days. On a smaller plan, intervals stay wide for longer,
    and rekkal will keep telling you so rather than pretending otherwise.
  </Accordion>

  <Accordion title="Which model answered is not knowable, and rekkal does not guess">
    This is the honest cost of reading the consumer interface rather than an API. The
    platform decides which model serves a signed-out session and publishes nothing about the
    choice, so there is no model version to record. Every answer on the evidence page says
    "Not disclosed" in those words, and names whose limitation it is. Nothing anywhere in
    rekkal fills that gap with a family name or a guess.

    A consequence worth stating: a week-over-week move can be a routing change rather than a
    visibility change, and rekkal cannot tell you which. It is a real limit of measuring a
    surface that routes.
  </Accordion>

  <Accordion title="Whether the platform searched is observed, not requested">
    rekkal used to turn grounding on and could therefore assert it. On a consumer interface
    the platform decides, so rekkal records what it *saw*: whether that answer cited sources.
    Each answer carries that observation, and each slice reports the fraction of its answers
    that searched.

    Where a surface gives no way to tell, it is recorded as undetermined — never as "did not
    search", which would be an assertion nobody established.
  </Accordion>

  <Accordion title="Market and language are where rekkal asked from, not what it asked for">
    On an API these were request parameters. On a consumer interface they are properties of
    the exit address and the interface locale, which is a weaker claim, and the evidence page
    words it that way rather than keeping the stronger one.
  </Accordion>

  <Accordion title="An interface can change, and then a week is missing rather than wrong">
    These interfaces are rewritten often, and when one changes in a way rekkal cannot read,
    that engine stops producing answers until it is fixed. What you will see is a **gap** — no
    point on the chart, no figure, and a note naming the engine. You will never see a zero
    standing in for an engine that failed, and no line is drawn across the gap.

    Where a week's answers were collected but too many attempts were blocked to trust the
    survivors, that slice is reported as unmeasured rather than measured on what got through.
    Blocking is not random: it clusters by address, by hour and plausibly by question, so a
    sample that survived it is not a sample of the same thing.
  </Accordion>

  <Accordion title="Mentioned is not yet distinguished from recommended">
    A mention is counted whether the answer recommends you or lists you in passing. The
    distinction matters and buyers ask for it; it is not built yet, and rekkal does not
    imply it has it.
  </Accordion>

  <Accordion title="Only the visibility figure opens onto its answers">
    rekkal stores every answer and the exact position of every mention in it, and the
    visibility card links straight to the answers it was counted from. Share of voice and
    average position do not have that link yet — they are counted over the same stored
    answers, but there is no screen that walks you through those two one answer at a time.
  </Accordion>
</AccordionGroup>

## Reproducing our numbers

The rules on this page are enforced in code and covered by tests, not by editorial
discipline: the suppression rule has one implementation, the read layer contains no
zero-filling, and the plan figures in these docs are checked against the values the product
enforces. If you want a specific figure explained, ask — showing the math is the point.