Robots.txt Distribución de uso en Islas Cocos

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Googlebot Disallow	31.420	14,97
Baidu Bot Disallow	31.401	14,96
Some Robots Disallow	19.050	9,08
Yandex Bot Disallow	15.243	7,26
Microsoft BingBot Disallow	14.598	6,96
Yahoo Slurp Disallow	14.290	6,81
Alexa Bot Disallow	13.089	6,24
Sogou Bot Disallow	12.919	6,16
Soso Bot Disallow	11.965	5,7
Robots Disallow	9.873	4,7
Ahrefs Bot Disallow	5.768	2,75
Majestic Bot Disallow	5.159	2,46
Semrush Bot Disallow	4.266	2,03
360 Bot Disallow	2.666	1,27
Google Media Partners Disallow	2.149 ^*	1,02
Google AdsBot Disallow	1.998	0,95
Microsoft MSNBot Disallow	1.978	0,94
Common Crawl Bot Disallow	1.171	0,56
Google AdsBot Mobile Disallow	1.104	0,53
GPTBot Disallow	1.010	0,48
AppleBot Disallow	826	0,39
Google Image Bot Disallow	784	0,37
Anthropic Claude Bot Disallow	659	0,31
ByteDance Bytespider Disallow	648	0,31
Facebook Bot Disallow	634	0,3
Google Bard Disallow	603	0,29
Exalead Bot Disallow	600	0,29
Youdao Bot Disallow	529	0,25
Google AdsBot Mobile Apps Disallow	503	0,24
Apple AI Bot Disallow	485	0,23
Cohere AI Disallow	416	0,2
Perplexity	400	0,19
BuiltWithBot Disallow	242	0,12
Webz Omgili Disallow	227	0,11
DuckDuckGo Bot Disallow	174	0,08
Google News Bot Disallow	167	0,08
Google Video Bot Disallow	162	0,08
Google Favicon Disallow	154	0,07
Google API Bot Disallow	143	0,07
Google DuplexWeb Disallow	140	0,07
YOU Disallow	85	0,04
Google FeedFetcher Disallow	83	0,04
Diffbot Disallow	80	0,04
Pipl Disallow	29	0,01
Google Read Aloud Disallow	1	0