Robots.txt Répartition de l'utilisation dans Guatemala

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Google AdsBot Disallow	1 834	16,56
Google AdsBot Mobile Disallow	1 342	12,12
Googlebot Disallow	1 133	10,23
Ahrefs Bot Disallow	579	5,23
GPTBot Disallow	465	4,2
Common Crawl Bot Disallow	459	4,14
AppleBot Disallow	454	4,1
Anthropic Claude Bot Disallow	450	4,06
Google Bard Disallow	450	4,06
Facebook Bot Disallow	448	4,05
Apple AI Bot Disallow	445	4,02
Perplexity	432	3,9
Cohere AI Disallow	429	3,87
Google AdsBot Mobile Apps Disallow	423	3,82
Robots Disallow	419	3,78
Yandex Bot Disallow	167	1,51
Baidu Bot Disallow	146	1,32
Some Robots Disallow	139	1,26
Semrush Bot Disallow	107	0,97
Sogou Bot Disallow	106	0,96
Microsoft BingBot Disallow	93	0,84
Alexa Bot Disallow	92	0,83
Yahoo Slurp Disallow	90	0,81
Google Media Partners Disallow	85 ^*	0,77
Soso Bot Disallow	82	0,74
Majestic Bot Disallow	55	0,5
Microsoft MSNBot Disallow	39	0,35
Webz Omgili Disallow	30	0,27
ByteDance Bytespider Disallow	20	0,18
Exalead Bot Disallow	10	0,09
Google Image Bot Disallow	10	0,09
Diffbot Disallow	9	0,08
YOU Disallow	7	0,06
360 Bot Disallow	5	0,05
BuiltWithBot Disallow	4	0,04
Google News Bot Disallow	2	0,02
Google Video Bot Disallow	2	0,02
DuckDuckGo Bot Disallow	2	0,02
Google Favicon Disallow	2	0,02
Youdao Bot Disallow	2	0,02
Google API Bot Disallow	2	0,02
Google DuplexWeb Disallow	2	0,02
Pipl Disallow	2	0,02