Robots.txt Usage Distribution in Aruba

Top In Robots.txt Usage Distribution in Aruba

Technology	Websites	%
Google AdsBot Disallow	243	26.05
Googlebot Disallow	214	22.94
Robots Disallow	38	4.07
GPTBot Disallow	31	3.32
Common Crawl Bot Disallow	30	3.22
Google Bard Disallow	29	3.11
Google AdsBot Mobile Apps Disallow	29	3.11
AppleBot Disallow	28	3
Anthropic Claude Bot Disallow	28	3
Facebook Bot Disallow	28	3
Apple AI Bot Disallow	28	3
Perplexity	28	3
Ahrefs Bot Disallow	27	2.89
Cohere AI Disallow	27	2.89
Some Robots Disallow	21	2.25
Yandex Bot Disallow	18	1.93
Google Media Partners Disallow	16 *	1.71
Majestic Bot Disallow	15	1.61
Semrush Bot Disallow	14	1.5
Microsoft BingBot Disallow	8	0.86
Alexa Bot Disallow	8	0.86
Yahoo Slurp Disallow	6	0.64
Sogou Bot Disallow	6	0.64
Soso Bot Disallow	5	0.54
Exalead Bot Disallow	2	0.21
Microsoft MSNBot Disallow	2	0.21
ByteDance Bytespider Disallow	1	0.11
Google Image Bot Disallow	1	0.11
360 Bot Disallow	1	0.11
Webz Omgili Disallow	1	0.11

933 Detections

of Robots.txt in Aruba. Last updated 14th Nov 2024.

Google AdsBot Disallow is currently the most popular technology in this category.