Robots.txt Répartition des utilisations aux Îles Vierges américaines

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Google AdsBot Disallow	473	20,57
Googlebot Disallow	327	14,22
Anthropic Claude Bot Disallow	107	4,65
Common Crawl Bot Disallow	106	4,61
GPTBot Disallow	104	4,52
Facebook Bot Disallow	103	4,48
AppleBot Disallow	102	4,44
Google Bard Disallow	102	4,44
Apple AI Bot Disallow	99	4,31
Cohere AI Disallow	98	4,26
Perplexity	97	4,22
Google AdsBot Mobile Apps Disallow	95	4,13
Ahrefs Bot Disallow	72	3,13
Baidu Bot Disallow	67	2,91
Robots Disallow	47	2,04
Yandex Bot Disallow	41	1,78
Some Robots Disallow	34	1,48
Sogou Bot Disallow	29	1,26
Majestic Bot Disallow	29	1,26
Alexa Bot Disallow	27	1,17
Semrush Bot Disallow	25	1,09
Microsoft BingBot Disallow	24	1,04
Yahoo Slurp Disallow	23	1
Soso Bot Disallow	20	0,87
Google Media Partners Disallow	15 ^*	0,65
Microsoft MSNBot Disallow	6	0,26
ByteDance Bytespider Disallow	5	0,22
Exalead Bot Disallow	5	0,22
Webz Omgili Disallow	4	0,17
Google Image Bot Disallow	4	0,17
360 Bot Disallow	3	0,13
Diffbot Disallow	2	0,09
BuiltWithBot Disallow	2	0,09
YOU Disallow	1	0,04
Pipl Disallow	1	0,04