Robots.txt Usage Distribution in Jamaica

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Googlebot Disallow	172	9.32
Ahrefs Bot Disallow	156	8.46
Robots Disallow	108	5.85
GPTBot Disallow	107	5.8
Common Crawl Bot Disallow	105	5.69
Google AdsBot Mobile Apps Disallow	101	5.47
Anthropic Claude Bot Disallow	97	5.26
AppleBot Disallow	96	5.2
Google Bard Disallow	96	5.2
Facebook Bot Disallow	95	5.15
Apple AI Bot Disallow	95	5.15
Cohere AI Disallow	95	5.15
Perplexity	94	5.09
Baidu Bot Disallow	73	3.96
Some Robots Disallow	51	2.76
Yandex Bot Disallow	41	2.22
Yahoo Slurp Disallow	41	2.22
Microsoft BingBot Disallow	37	2.01
Alexa Bot Disallow	33	1.79
Sogou Bot Disallow	30	1.63
Soso Bot Disallow	30	1.63
Majestic Bot Disallow	18	0.98
Google Media Partners Disallow	16 ^*	0.87
Microsoft MSNBot Disallow	16	0.87
Google Image Bot Disallow	14	0.76
Semrush Bot Disallow	11	0.6
Exalead Bot Disallow	5	0.27
Google News Bot Disallow	3	0.16
Google Video Bot Disallow	3	0.16
ByteDance Bytespider Disallow	1	0.05
360 Bot Disallow	1	0.05
Webz Omgili Disallow	1	0.05
BuiltWithBot Disallow	1	0.05
Diffbot Disallow	1	0.05
Youdao Bot Disallow	1	0.05