How do AI visibility tools collect answers, and can I trust the numbers?

Ask how the answers were collected, because it decides how far the numbers can be trusted.

Collection methods vary. Tools can use an engine's official API, a licensed data provider, or automated browsing of the public interface. Each has different coverage, cost, and fidelity to what a real user sees. Doily collects AI answers through official APIs and licensed data providers.

Questions worth asking any vendor

1. How are answers collected? Official API, licensed provider, or something else.
2. Is it the web-grounded version? An engine's plain model can answer differently from the version that searches the web and cites sources.
3. What settings are used? Location, language, and whether the account is logged in.
4. How often are prompts run? And is the cadence shown next to the numbers? Doily's paid plans check prompts daily, free checks run weekly, and the product prints the cadence next to each engine's numbers.
5. Can you read the full answers and sources? A score with no raw answer behind it is hard to verify.
6. Which results are approximate? In Doily, DeepSeek is labeled approximate because it runs as open weights on a U.S. cloud, grounded with Google results.

How to read the numbers

• Treat each run as a sample. Look at the rate across many runs and the trend.
• Compare like with like: the same prompts, engines, and location.
• Use cited sources and the full answers to check the story behind a number.
• Do not compare numbers across tools. Methods differ.

If a tool cannot tell you how it collects answers, or hides which results are estimates, treat the numbers as directional at best.

Check a few prompts by hand against the tool's output. They will not match exactly, because answers vary, but the brands and sources should be in the same range.

A manual spot check.

Pick 5 prompts, run them by hand on the same engine and location the tool uses, and compare. The answers will not match word for word, because answers vary. Compare the brands named and the sources cited.

If the tool shows brands or sources you never see by hand, or misses ones you always see, ask the vendor why.

Ask about history and export.

Trends need weeks of data. Ask how long answers are kept, whether you can read the full text of each answer, and whether you can export them. A number you cannot trace back to an answer is hard to defend in front of a client.