Robots.txt Usage Distribution in Guam

Distribution for websites using Robots.txt technologies

Technology	Websites	%
GPTBot Disallow	40	8.66
Common Crawl Bot Disallow	38	8.23
Anthropic Claude Bot Disallow	38	8.23
Google Bard Disallow	37	8.01
AppleBot Disallow	35	7.58
Facebook Bot Disallow	35	7.58
Apple AI Bot Disallow	35	7.58
Cohere AI Disallow	35	7.58
Perplexity	35	7.58
Google AdsBot Mobile Apps Disallow	34	7.36
Ahrefs Bot Disallow	33	7.14
Yandex Bot Disallow	17	3.68
Robots Disallow	9	1.95
Semrush Bot Disallow	9	1.95
Yahoo Slurp Disallow	5	1.08
Majestic Bot Disallow	4	0.87
ByteDance Bytespider Disallow	4	0.87
Alexa Bot Disallow	3	0.65
Webz Omgili Disallow	3	0.65
DuckDuckGo Bot Disallow	3	0.65
Microsoft BingBot Disallow	2	0.43
Sogou Bot Disallow	2	0.43
Soso Bot Disallow	1	0.22
Microsoft MSNBot Disallow	1	0.22
Google Image Bot Disallow	1	0.22
Diffbot Disallow	1	0.22
YOU Disallow	1	0.22
Pipl Disallow	1	0.22