The same prompt gives a different answer every time. Which result is real?

Both are real, and neither is the number. An AI answer is generated fresh each time, so a single run is a sample, not a measurement.

Several things move an answer around: the exact wording of the prompt, the location of the person asking, the model version behind the engine, which sources the engine retrieves that day, and sometimes the user's own history. Two runs a minute apart can differ.

What you measure instead is a rate. Say you run a prompt 10 times and you are mentioned in 3 of them. Your mention rate for that prompt is 30%. Next month you run it 10 times again and get 5 of 10. That is a trend worth watching. One bad Wednesday is not.

How to set it up so the numbers mean something:

1. Fix a prompt set. Write 20 to 30 prompts in your customers' words and keep them stable.
2. Fix the settings. Same engine, same location, same schedule.
3. Run them on a regular cadence. Doily checks paid plans' prompts daily and free checks weekly, and prints the cadence next to each engine's numbers.
4. Read the rate across runs, per prompt and per engine, not the latest answer.
5. Look at which sources keep showing up. Stable sources tell you more than a stable ranking order.

When a prompt flips between mentioned and absent, it usually means you are on the edge. Those are the prompts where a better page or a new citation can tip the result, so they deserve the most attention.

How many runs do you need?

There is no number that works for every site. Pick one you can keep up, run the same prompts on the same schedule, and report counts along with percentages. "3 of 10" shows how thin the sample is, and "30%" does not.

With a small sample, a gap of a few points between two periods or two brands is noise. Look for movement that holds across several periods before you act on it.

Hold the setup constant.

Keep the engine, the prompt wording, the location, and the schedule fixed, and write down anything else you cannot control. Google says location and settings affect Search results, and OpenAI's web search tool takes an approximate location as an input, so a local prompt needs a stated place.

We have not found a study that isolates device effects on AI answers. Log the device and keep it the same.