
AI crawlers now account for roughly 4.2% of all HTML requests hitting websites globally, according to Cloudflare Radar data. But that single number hides a dramatic geographic divide. I analyzed 30 days of Cloudflare Radar AI bot traffic data across 11 countries and found that the dominant crawler, its purpose, and its intensity vary wildly depending on where your website's audience sits. In Western markets, Googlebot commands nearly half of all AI-related traffic. In Brazil, Japan, and Australia, ChatGPT-User is the runaway leader -- and the crawling purpose shifts from model training to real-time user queries.
Globally, Googlebot leads all AI-related crawler traffic with a 38.7% share, according to Cloudflare Radar AI Insights (get_ai_data endpoint). GPTBot follows at 12.8%, Meta-ExternalAgent at 11.6%, and ClaudeBot at 11.4%. These four crawlers alone represent 74.3% of all identified AI bot requests hitting websites protected by Cloudflare's network of 330+ cities in 125+ countries.
| AI Bot | Global Share (%) | Operator | Primary Purpose |
|---|---|---|---|
| Googlebot | 38.7% | Search indexing + AI training (mixed) | |
| GPTBot | 12.8% | OpenAI | Model training |
| Meta-ExternalAgent | 11.6% | Meta | AI training |
| ClaudeBot | 11.4% | Anthropic | Model training |
| Bingbot | 9.7% | Microsoft | Search indexing + AI (mixed) |
| Amazonbot | 4.8% | Amazon | AI training |
| Bytespider | 3.5% | ByteDance | AI training |
| Applebot | 2.5% | Apple | Search + AI features |
| OAI-SearchBot | 2.0% | OpenAI | ChatGPT Search |
One significant month-over-month shift stands out: Meta-ExternalAgent surged from 8.5% to 11.6% of global AI bot traffic between December 2025 and January 2026 -- a 36% relative increase in just 30 days. Meanwhile, Googlebot's share declined from 42.6% to 38.7%, suggesting Meta is ramping up its training data collection aggressively while Google's proportional footprint shrinks as competitors grow.
The most striking finding in Cloudflare Radar's country-level data is a clean geographic split between two patterns. I'm calling them the "Googlebot Belt" and the "ChatGPT Corridor."
In the United States, United Kingdom, Germany, France, Netherlands, and Canada, Googlebot holds the #1 position among AI crawlers. France stands out as the most Googlebot-heavy country I analyzed, with Googlebot commanding 63.6% of all AI bot traffic -- nearly two-thirds of every AI-related request hitting French websites.
| Country | Googlebot Share | #2 Bot | #2 Share | Training Purpose % |
|---|---|---|---|---|
| France | 63.6% | ClaudeBot | 7.8% | 25.7% |
| Germany | 47.8% | Bingbot | 10.3% | 30.6% |
| Canada | 47.1% | imgproxy | 22.4% | 17.2% |
| Netherlands | 45.7% | GPTBot | 10.5% | 33.7% |
| United States | 40.3% | GPTBot | 13.6% | 41.0% |
| United Kingdom | 38.5% | Bingbot | 18.8% | 28.1% |
According to Cloudflare Radar regional data, the United States has the highest concentration of pure training crawlers (41.0% of AI bot traffic classified as "Training"), which aligns with the US being home to OpenAI, Anthropic, Meta AI, and Amazon's AI labs. The UK shows an unusually high Bingbot presence at 18.8%, nearly double the global average of 9.7%.
In Japan, Brazil, Australia, South Korea, and India, ChatGPT-User -- the bot that visits pages when real users ask ChatGPT questions -- is the dominant AI crawler. This is a fundamentally different pattern. Instead of companies training models, what I'm seeing is end-user demand for AI-powered answers driving the crawl traffic.
| Country | ChatGPT-User Share | #2 Bot | #2 Share | User Action Purpose % |
|---|---|---|---|---|
| Australia | 75.8% | Googlebot | 10.2% | 76.0% |
| Brazil | 57.6% | Googlebot | 26.1% | 57.7% |
| Japan | 46.2% | Googlebot | 19.4% | 46.5% |
| South Korea | 43.9% | Googlebot | 24.4% | 44.1% |
| India | 39.8% | Googlebot | 37.3% | 40.2% |
Australia is the single most extreme case in my analysis. A staggering 75.8% of all AI bot traffic hitting Australian websites comes from ChatGPT-User, with 76.0% of crawl activity classified as "User Action." This means three out of every four AI crawler requests to Australian sites are triggered by someone actively asking ChatGPT a question -- not by a training pipeline scraping content.
The Googlebot Belt vs. ChatGPT Corridor divide likely reflects three overlapping factors I've identified from the data:
According to Cloudflare Radar AI Insights (get_ai_data industry dimension), retail and e-commerce sites absorb the largest share of AI crawling activity globally. Shopping-related content is the single most crawled vertical, followed by technology and news media.
| Industry Vertical | Share of AI Bot Traffic |
|---|---|
| Shopping and General Merchandise | 31.2% |
| Internet and Telecom | 17.0% |
| Computer and Electronics | 15.0% |
| News, Media, and Publications | 8.9% |
| Business and Industry | 5.2% |
| Travel and Tourism | 3.8% |
| Professional Services | 3.5% |
| Gambling | 3.1% |
| Finance | 3.0% |
The retail dominance at 31.2% makes sense when you consider how AI models are increasingly used for product research and comparison shopping. When a user asks ChatGPT "what's the best running shoe under $150," the bot needs to crawl current product pages and reviews to generate its answer. This is a direct bandwidth cost that e-commerce operators should factor into their infrastructure planning.
Cloudflare Radar classifies AI bot activity into four purpose categories. Globally, the split reveals that nearly half of all AI crawling serves a dual or ambiguous purpose:
The country-level crawl purpose data reveals sharp contrasts. In the United States, 91.6% of AI bot activity is classified as either Mixed Purpose or Training -- meaning US websites serve primarily as raw material for model building. In Brazil, the picture inverts: 57.7% of AI bot activity is User Action, meaning the majority of crawls are triggered by actual people asking AI for answers about Brazilian content.
Comparing Cloudflare Radar data from January 9 - February 8, 2026 against the prior 30-day control period (December 10, 2025 - January 9, 2026) reveals several notable shifts in the global AI crawler market:
| AI Bot | Previous 30d Share | Current 30d Share | Change |
|---|---|---|---|
| Googlebot | 42.6% | 38.7% | -3.9 pp |
| Meta-ExternalAgent | 8.5% | 11.6% | +3.1 pp |
| GPTBot | 12.9% | 12.8% | -0.1 pp |
| ClaudeBot | 13.0% | 11.4% | -1.6 pp |
| Bingbot | 9.4% | 9.7% | +0.3 pp |
| Bytespider | 2.9% | 3.5% | +0.6 pp |
| OAI-SearchBot | 1.5% | 2.0% | +0.5 pp |
Meta's training crawler grew its share by 36% in a single month. That's a significant ramp-up and aligns with Meta's public statements about expanding its Llama model training pipeline. OAI-SearchBot's climb from 1.5% to 2.0% is also worth noting -- a 33% relative increase that reflects growing usage of ChatGPT's search feature.
Based on what I've found in the Cloudflare Radar data, here are specific actions depending on your situation:
If your audience is in the US or Western Europe:
If your audience is in Japan, Brazil, Australia, or South Korea:
For e-commerce and retail sites everywhere:
This analysis uses data from Cloudflare Radar's AI Insights endpoint (get_ai_data), which aggregates traffic patterns across Cloudflare's global network spanning 330 cities in 125+ countries. The data covers January 9 through February 8, 2026.
I queried bot traffic breakdowns by user agent, crawl purpose, industry, and vertical across 11 countries: United States, United Kingdom, Germany, Japan, India, Brazil, France, Australia, Canada, South Korea, and Netherlands. Month-over-month comparison uses a 30-day control window (December 10, 2025 - January 9, 2026). All percentages represent share of identified AI bot requests, not share of total web traffic.
Cloudflare's network processes over 81 million HTTP requests per second and 67 million DNS queries per second, providing one of the most comprehensive views of global internet traffic available. The location filter corresponds to the billing country of the Cloudflare customer whose site received the traffic.
Data source: Cloudflare Radar AI Insights - get_ai_data endpoint (radar.cloudflare.com). Last updated: February 8, 2026.
