Robots.txt Usage Distribution in Honduras

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Google AdsBot Disallow	710	15.51
Googlebot Disallow	483	10.55
Google AdsBot Mobile Disallow	475	10.37
Ahrefs Bot Disallow	265	5.79
Robots Disallow	176	3.84
Common Crawl Bot Disallow	158	3.45
GPTBot Disallow	150	3.28
AppleBot Disallow	145	3.17
Anthropic Claude Bot Disallow	145	3.17
Google Bard Disallow	145	3.17
Facebook Bot Disallow	144	3.14
Apple AI Bot Disallow	142	3.1
Cohere AI Disallow	138	3.01
Perplexity	138	3.01
Google AdsBot Mobile Apps Disallow	137	2.99
Baidu Bot Disallow	119	2.6
Google Media Partners Disallow	113 ^*	2.47
Sogou Bot Disallow	102	2.23
Yandex Bot Disallow	98	2.14
Alexa Bot Disallow	92	2.01
Microsoft BingBot Disallow	88	1.92
Yahoo Slurp Disallow	87	1.9
Soso Bot Disallow	82	1.79
Some Robots Disallow	77	1.68
Majestic Bot Disallow	39	0.85
Semrush Bot Disallow	33	0.72
Microsoft MSNBot Disallow	16	0.35
Exalead Bot Disallow	12	0.26
360 Bot Disallow	12	0.26
Webz Omgili Disallow	11	0.24
Google Image Bot Disallow	10	0.22
ByteDance Bytespider Disallow	9	0.2
BuiltWithBot Disallow	8	0.17
Youdao Bot Disallow	4	0.09
YOU Disallow	3	0.07
DuckDuckGo Bot Disallow	3	0.07
Google News Bot Disallow	2	0.04
Google Video Bot Disallow	2	0.04
Google Favicon Disallow	2	0.04
Google API Bot Disallow	2	0.04
Google DuplexWeb Disallow	2	0.04