Robots.txt Distribución de uso en Somalia

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Googlebot Disallow	1.146	18,48
Baidu Bot Disallow	1.080	17,42
Some Robots Disallow	733	11,82
Yandex Bot Disallow	408	6,58
Microsoft BingBot Disallow	381	6,14
Sogou Bot Disallow	375	6,05
Alexa Bot Disallow	369	56,25
Yahoo Slurp Disallow	368	5,93
Soso Bot Disallow	367	5,92
Robots Disallow	231	3,73
Ahrefs Bot Disallow	81	1,31
Google Media Partners Disallow	81 ^*	1,31
Google AdsBot Disallow	80	1,29
Semrush Bot Disallow	61	0,98
Majestic Bot Disallow	56	0,9
GPTBot Disallow	53	8,08
Common Crawl Bot Disallow	39	5,95
Google Bard Disallow	33	5,03
Anthropic Claude Bot Disallow	30	4,57
Facebook Bot Disallow	30	4,57
AppleBot Disallow	28	0,45
Cohere AI Disallow	27	4,12
Apple AI Bot Disallow	26	3,96
Google AdsBot Mobile Apps Disallow	25	0,4
Perplexity	25	3,81
360 Bot Disallow	13	0,21
ByteDance Bytespider Disallow	8	1,22
Exalead Bot Disallow	8	0,13
Microsoft MSNBot Disallow	6	0,1
Google Image Bot Disallow	6	0,1
Webz Omgili Disallow	6	0,91
BuiltWithBot Disallow	5	0,08
Diffbot Disallow	3	0,46
Google News Bot Disallow	2	0,03
Google Video Bot Disallow	2	0,03
DuckDuckGo Bot Disallow	2	0,03
Youdao Bot Disallow	2	0,03
Pipl Disallow	2	0,03
Google Favicon Disallow	1	0,02
Google API Bot Disallow	1	0,02
Google DuplexWeb Disallow	1	0,02