Robots.txt Usage Distribution in Ethiopia

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Googlebot Disallow	324	9.99
Microsoft BingBot Disallow	243	7.49
Yandex Bot Disallow	235	7.25
Yahoo Slurp Disallow	230	7.09
Soso Bot Disallow	225	6.94
Alexa Bot Disallow	224	6.91
Baidu Bot Disallow	217	6.69
Sogou Bot Disallow	195	6.01
Robots Disallow	189	5.83
Google Media Partners Disallow	132 ^*	4.07
GPTBot Disallow	96	2.96
Common Crawl Bot Disallow	81	2.5
Google Bard Disallow	74	2.28
Anthropic Claude Bot Disallow	72	2.22
Facebook Bot Disallow	72	2.22
Perplexity	72	2.22
AppleBot Disallow	71	2.19
Cohere AI Disallow	71	2.19
Apple AI Bot Disallow	69	2.13
Some Robots Disallow	64	1.97
Google AdsBot Mobile Apps Disallow	58	1.79
Ahrefs Bot Disallow	49	1.51
ByteDance Bytespider Disallow	39	1.2
Semrush Bot Disallow	36	1.11
Majestic Bot Disallow	16	0.49
Webz Omgili Disallow	15	0.46
Diffbot Disallow	15	0.46
YOU Disallow	13	0.4
Microsoft MSNBot Disallow	10	0.31
Exalead Bot Disallow	8	0.25
360 Bot Disallow	6	0.19
Google Image Bot Disallow	4	0.12
BuiltWithBot Disallow	4	0.12
Google News Bot Disallow	2	0.06
Google Video Bot Disallow	2	0.06
DuckDuckGo Bot Disallow	2	0.06
Google Favicon Disallow	2	0.06
Google API Bot Disallow	2	0.06
Google DuplexWeb Disallow	2	0.06
Youdao Bot Disallow	1	0.03
Pipl Disallow	1	0.03