AI models often provide generic descriptions of products because they process only fragments of a webpage rather than the entire site. Research by Vercel and MERJ indicates that crawlers for ChatGPT and Claude download HTML but do not execute JavaScript, meaning content rendered dynamically in a browser remains invisible to them.
A study by Profound on 77 public pricing pages found that robots failed to see at least 40% of the main text on approximately one-eighth of the sites. When pricing information is missing from the source HTML, AI systems may pull data from third-party sources, leading to significant discrepancies in reported costs.
Ahrefs tested five major AI systems, including ChatGPT, Claude, and Gemini, and found that they ignore structured data formats like JSON-LD, Microdata, and RDFa. Consequently, critical product details such as pricing and target audience must be explicitly included in the visible text of the source code.
Anthropic suggests that adding context, such as document titles, to text fragments can reduce search errors. For website owners, this means ensuring that every paragraph on a product page clearly identifies the product, its purpose, and its target audience, regardless of the surrounding context.
