The author analyzed 358 AI responses to 20 questions for a client to evaluate brand visibility. An initial automated scan showed a 15% visibility rate, as the company was mentioned in 3 out of 20 questions. However, manual review revealed that these results were misleading due to the script's simple keyword matching.
One positive result occurred when GPT-4o provided a link to the company's article but attributed the methodology to another contractor. Another instance involved Perplexity citing company pages as sources for pricing and reviews without actually recommending the brand. In a third case, the brand name was already present in the user's prompt, which skewed the results.
The analysis showed that Claude sometimes confused the company with a music distributor or a telephony service, or simply stated it could not find the company, despite the script marking these as positive hits. After refining the methodology to exclude prompts containing the brand name and verifying actual recommendations, the visibility rate dropped from 15% to 5.3%.
The final result confirmed that Alice AI correctly recommended the company and described its services in three separate responses to a single question. The author emphasizes that citations and links are different from business recommendations and should be tracked separately in AI performance reports.
To improve accuracy, the author suggests categorizing questions by user intent—such as understanding a service, estimating budget, or finding a contractor—and using specific metadata like prompt_id and brand_in_prompt to avoid counting repetitions as unique choice situations.