Robots.txt Usage Distribution in Sint Maarten

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Googlebot Disallow	201	15.36
Microsoft BingBot Disallow	161	12.3
Yandex Bot Disallow	157	11.99
Alexa Bot Disallow	149	64.22
Yahoo Slurp Disallow	148	11.31
Sogou Bot Disallow	145	11.08
Soso Bot Disallow	145	11.08
Robots Disallow	75	5.73
GPTBot Disallow	14	6.03
Anthropic Claude Bot Disallow	10	4.31
Perplexity	10	4.31
Google Bard Disallow	9	3.88
Common Crawl Bot Disallow	8	3.45
AppleBot Disallow	8	0.61
Facebook Bot Disallow	8	3.45
Google AdsBot Mobile Apps Disallow	8	0.61
Apple AI Bot Disallow	8	3.45
Cohere AI Disallow	8	3.45
Semrush Bot Disallow	7	0.53
Ahrefs Bot Disallow	5	0.38
Majestic Bot Disallow	4	0.31
Microsoft MSNBot Disallow	3	0.23
ByteDance Bytespider Disallow	2	0.86
Google Image Bot Disallow	2	0.15
Webz Omgili Disallow	2	0.86
Google News Bot Disallow	2	0.15
Google Video Bot Disallow	2	0.15
DuckDuckGo Bot Disallow	2	0.15
Google Favicon Disallow	2	0.15
Google API Bot Disallow	2	0.15
Google DuplexWeb Disallow	2	0.15