We asked ChatGPT for the best insurance agency in Albany three times. It named 15 different firms, with none in common.

Ask Google for insurance near you and you get a map with three businesses on it. Ask again tomorrow and it is the same three. That stability is what makes local search optimisable: there is a result to compete for, and holding it means something.

We assumed the AI assistants worked the same way, and set out to build a board showing which agency ChatGPT and Gemini recommend in each of the 109 metros we measure.

The board does not exist, because the answer does not hold still.

What we did

Each metro was queried with a generic prompt, best insurance agency near me, with the city supplied as a location rather than written into the question. That matters: it measures what someone in Omaha actually gets, not what a model says when you name Omaha at it.

• Gemini, geo-targeted to city level, 109 metros, three keywords, three runs each: 981 captures
• ChatGPT, geo-targeted to city level through OpenAI’s own API, 109 metros, three runs each: 327 captures

Then we counted how much the recommendations overlapped between runs of the identical query.

The result

Comparing like with like, the same keyword on both engines:

Overlap is measured as the share of businesses common to all three runs against every business named across them. A score of 1.0 means the same answer every time. A score of 0 means nothing survived all three.

ChatGPT has a median of zero: in the typical metro, not one business it named survived all three runs. Gemini is better, but only in the sense that roughly one business in eleven persists.

Across all three keywords Gemini ran, the picture is the same: 198 metro-and-keyword pairs, mean overlap 0.128, and 95 of them with nothing recurring at all.

What that looks like in one city

Albany, three identical queries to ChatGPT, minutes apart:

• 15 distinct agencies named
• Not one appeared in all three answers

Los Angeles: 13 distinct, none recurring. Atlanta: 12 distinct, none recurring. Baltimore, Denver, Dallas, Vancouver and Winnipeg: nothing recurring either.

Every individual answer looks authoritative. Five to seven named local agencies, with addresses, specialisms and a line on why each is worth calling. Nothing in the answer tells you it would have named a different five if you had asked again.

The bigger the market, the less stable the answer

The metros with any measurable overlap are the ones where the model named fewer businesses to begin with. Lansing produced the shortest list, five distinct firms across three runs, and the highest overlap at 0.60. Washington also named five, at 0.20.

At the other end, every one of the largest markets returned nothing recurring: New York, Los Angeles, Chicago, Dallas, Denver, Vancouver, Winnipeg, San Francisco, Seattle’s neighbours and more. Fifty-nine of 109 metros in total.

That is not the model being more confident about smaller cities. It is a shorter pool to draw from, so repeats happen by chance more often. In a large market the pool is deep enough that three samples rarely intersect.

Why this matters if you sell insurance locally

You cannot be “the agency AI recommends.” There is no such position. In the same city, in the same minute, the answer names a different set of firms depending on nothing you can observe or influence.

You cannot verify a claim that you hold one. If someone shows you a screenshot of ChatGPT recommending their client, that screenshot has roughly the evidentiary weight of a single coin flip. Ours would have looked equally convincing, and we have seventeen of them for Albany alone.

Anything sold to you as AI local visibility optimisation is being sold against this. Optimising toward a result that does not persist between two consecutive queries is not a strategy. It may become one. It is not one now.

Meanwhile the map pack is exactly as measurable as it has always been. Across the same 109 metros, 3,923 map slots, the same businesses hold the same positions month over month. 95.2% of the businesses holding a slot operate in a single city, and 68 of the 109 metros are led by a single-market operator. That is a contest with a scoreboard.

What we are publishing instead

Rather than naming a leader per metro, which would mean publishing a name the engine says about a third of the time, we will publish stability: how many distinct businesses each engine names across repeated runs, how many recur, and the overlap score.

The measurement becomes consistency rather than a name. It is the honest version, and for anyone deciding where to put effort it is the more useful one.

The caveats

This is one query type. Generic near-me intent for insurance. Other categories, and more specific prompts, may behave differently. We have not tested that.

The two engines are not perfectly matched. ChatGPT covers 109 metros on one keyword. Gemini covers 73 on that same keyword, because not every metro produced three usable runs on it. The comparison above uses the shared keyword only.

Three runs is a floor, not a ceiling. More runs might surface a stable core the way repeated sampling surfaces a mean. A median overlap of zero suggests there may be no core to find, but it does not prove it.

Extraction matters more than we expected. An early pass counted phrases like “What type of insurance are you looking for?” as business names, which inflated the distinct counts and depressed the overlap scores. The figures here come from a tightened filter, and both engines are measured with an identical one. Our first numbers were wrong and these replace them.

Models change without notice. Both engines could become deterministic tomorrow, and we would only know because we keep measuring. This figure is August 2026, not a permanent property of the technology.

The method

• Gemini captured through the DataForSEO LLM Scraper with city-level location targeting, three runs per metro per keyword
• ChatGPT captured through OpenAI’s Responses API with the web search tool and user_location set to city, region and country, three runs per metro
• Business names extracted from the formatted recommendation lists, with section headers and clarifying questions filtered out
• Overlap calculated as the intersection over the union of business sets across all runs of an identical query

Every raw response is archived. Anyone can rerun this: the prompts are one sentence and the cost is measured in cents.

The boards behind the Google figures are at https://insuranceseosem.com/index/local/, free, with the capture date on every number.