Robots.txt Distribución de uso en Perú

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Google AdsBot Disallow	5.207	15,8
Googlebot Disallow	3.891	11,81
Ahrefs Bot Disallow	3.332	10,11
Robots Disallow	2.269	6,89
Google AdsBot Mobile Disallow	2.129	6,46
Google Media Partners Disallow	1.961 ^*	5,95
Baidu Bot Disallow	1.293	3,92
Sogou Bot Disallow	919	2,79
Yandex Bot Disallow	906	2,75
Microsoft BingBot Disallow	840	2,55
Yahoo Slurp Disallow	823	2,5
Some Robots Disallow	805	2,44
Alexa Bot Disallow	793	2,41
Soso Bot Disallow	717	2,18
GPTBot Disallow	689	2,09
Common Crawl Bot Disallow	651	1,98
Google Bard Disallow	588	1,78
Anthropic Claude Bot Disallow	579	1,76
AppleBot Disallow	567	1,72
Facebook Bot Disallow	554	1,68
Apple AI Bot Disallow	537	1,63
Perplexity	529	1,61
Cohere AI Disallow	519	1,58
Google AdsBot Mobile Apps Disallow	507	1,54
Semrush Bot Disallow	323	0,98
Majestic Bot Disallow	305	0,93
Microsoft MSNBot Disallow	147	0,45
Google Image Bot Disallow	108	0,33
Exalead Bot Disallow	91	0,28
ByteDance Bytespider Disallow	72	0,22
Webz Omgili Disallow	60	0,18
360 Bot Disallow	46	0,14
Youdao Bot Disallow	33	0,1
BuiltWithBot Disallow	28	0,08
Google News Bot Disallow	24	0,07
Google Video Bot Disallow	24	0,07
Diffbot Disallow	22	0,07
YOU Disallow	18	0,05
DuckDuckGo Bot Disallow	12	0,04
Google Favicon Disallow	10	0,03
Google API Bot Disallow	7	0,02
Google DuplexWeb Disallow	7	0,02
Pipl Disallow	4	0,01
Google FeedFetcher Disallow	1	0