Robots.txt Usage Distribution in Sierra Leone

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Googlebot Disallow	85	21.04
Robots Disallow	57	14.11
GPTBot Disallow	26	16.56
Common Crawl Bot Disallow	19	12.1
Google Bard Disallow	19	12.1
Anthropic Claude Bot Disallow	18	11.46
Google AdsBot Mobile Apps Disallow	17	4.21
Cohere AI Disallow	17	10.83
Google Media Partners Disallow	16 ^*	3.96
Facebook Bot Disallow	16	10.19
AppleBot Disallow	15	3.71
Apple AI Bot Disallow	15	9.55
Perplexity	14	8.92
Yandex Bot Disallow	13	3.22
Ahrefs Bot Disallow	13	3.22
Majestic Bot Disallow	9	2.23
Semrush Bot Disallow	8	1.98
Microsoft BingBot Disallow	5	1.24
Alexa Bot Disallow	5	3.18
Yahoo Slurp Disallow	4	0.99
Sogou Bot Disallow	4	0.99
Soso Bot Disallow	4	0.99
ByteDance Bytespider Disallow	2	1.27
Microsoft MSNBot Disallow	1	0.25
Webz Omgili Disallow	1	0.64
Diffbot Disallow	1	0.64