Robots.txt Usage Distribution in Bolivia

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Googlebot Disallow	388	12.2
Google AdsBot Disallow	372	11.69
Robots Disallow	325	10.22
Google AdsBot Mobile Disallow	236	7.42
Ahrefs Bot Disallow	164	5.16
Baidu Bot Disallow	154	4.84
Alexa Bot Disallow	107	3.36
Sogou Bot Disallow	105	3.3
Yahoo Slurp Disallow	105	3.3
Yandex Bot Disallow	103	3.24
Some Robots Disallow	102	3.21
Microsoft BingBot Disallow	98	3.08
Soso Bot Disallow	93	2.92
Google Media Partners Disallow	81 ^*	2.55
GPTBot Disallow	66	2.07
Common Crawl Bot Disallow	63	1.98
Anthropic Claude Bot Disallow	57	1.79
Google Bard Disallow	53	1.67
AppleBot Disallow	52	1.63
Facebook Bot Disallow	51	1.6
Majestic Bot Disallow	51	1.6
Apple AI Bot Disallow	50	1.57
Cohere AI Disallow	50	1.57
Perplexity	50	1.57
Google AdsBot Mobile Apps Disallow	45	1.41
Semrush Bot Disallow	44	1.38
Microsoft MSNBot Disallow	31	0.97
Exalead Bot Disallow	18	0.57
ByteDance Bytespider Disallow	12	0.38
Google Image Bot Disallow	12	0.38
Webz Omgili Disallow	9	0.28
360 Bot Disallow	9	0.28
YOU Disallow	6	0.19
Youdao Bot Disallow	6	0.19
BuiltWithBot Disallow	5	0.16
Diffbot Disallow	3	0.09
Google News Bot Disallow	3	0.09
Pipl Disallow	2	0.06