Robots.txt Distribución de uso en Costa de Marfil

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Robots Disallow	289	18,8
Googlebot Disallow	204	13,27
Baidu Bot Disallow	97	6,31
Ahrefs Bot Disallow	84	5,47
Yandex Bot Disallow	68	4,42
Sogou Bot Disallow	62	4,03
Some Robots Disallow	61	3,97
Soso Bot Disallow	60	3,9
Microsoft BingBot Disallow	56	3,64
Yahoo Slurp Disallow	55	3,58
Alexa Bot Disallow	52	3,38
Common Crawl Bot Disallow	47	3,06
GPTBot Disallow	37	2,41
Google AdsBot Mobile Apps Disallow	31	2,02
Google Bard Disallow	28	1,82
Majestic Bot Disallow	28	1,82
Anthropic Claude Bot Disallow	27	1,76
Facebook Bot Disallow	27	1,76
Semrush Bot Disallow	27	1,76
AppleBot Disallow	26	1,69
Apple AI Bot Disallow	26	1,69
Cohere AI Disallow	26	1,69
Perplexity	25	1,63
Exalead Bot Disallow	15	0,98
360 Bot Disallow	12	0,78
BuiltWithBot Disallow	12	0,78
Google Image Bot Disallow	10	0,65
Microsoft MSNBot Disallow	8	0,52
Google News Bot Disallow	8	0,52
Google Video Bot Disallow	8	0,52
DuckDuckGo Bot Disallow	5	0,33
Google Favicon Disallow	5	0,33
Google API Bot Disallow	5	0,33
Google DuplexWeb Disallow	5	0,33
Webz Omgili Disallow	1	0,07