Cloudflare has introduced a default block on AI shopping agents and training bots for websites that display advertisements. The company maintains that ads signify content intended for human viewers, whereas AI agents often bypass these ads while extracting product data and reviews.
The new policy categorizes bot traffic into three distinct controls: search crawlers, AI training, and user-directed agents. While search bots remain permitted, AI training and shopping agents are restricted by default on ad-supported domains. This update applies to new domains, new sites from existing clients, and free-tier accounts.
Data from Cloudflare indicates that AI training accounted for 52% of crawler traffic on its network as of June, compared to 22% in the spring of 2025. Currently, 17% of site owners restrict AI training, while less than 1% block search bots. Cloudflare aims to eliminate mixed-use crawler traffic within one year.
Despite the growth in AI-assisted shopping, with 132 million U.S. adults having used AI for purchases, visibility remains low. Only 23% of merchants can identify both AI-driven traffic and the resulting sales. Meanwhile, retailers like Amazon have taken legal action against AI agents, while others explore integration.
