Robots.txt Distribución de uso en Marruecos

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Google AdsBot Disallow	1.962	16,93
Ahrefs Bot Disallow	1.371	11,83
Googlebot Disallow	1.215	10,48
Robots Disallow	1.094	9,44
Google AdsBot Mobile Disallow	772	6,66
Baidu Bot Disallow	436	3,76
Yandex Bot Disallow	314	2,71
Microsoft BingBot Disallow	287	2,48
Yahoo Slurp Disallow	287	2,48
Alexa Bot Disallow	284	13,69
Some Robots Disallow	279	2,41
Sogou Bot Disallow	259	2,23
GPTBot Disallow	258	12,43
Common Crawl Bot Disallow	252	12,14
Soso Bot Disallow	251	2,17
Google AdsBot Mobile Apps Disallow	220	1,9
Google Bard Disallow	218	10,51
Anthropic Claude Bot Disallow	215	10,36
AppleBot Disallow	206	1,78
Facebook Bot Disallow	206	9,93
Apple AI Bot Disallow	197	9,49
Cohere AI Disallow	197	9,49
Perplexity	194	9,35
Majestic Bot Disallow	143	1,23
Semrush Bot Disallow	124	1,07
Google Media Partners Disallow	65 ^*	0,56
Microsoft MSNBot Disallow	54	0,47
Google Image Bot Disallow	42	0,36
Exalead Bot Disallow	33	0,28
DuckDuckGo Bot Disallow	23	0,2
Google News Bot Disallow	19	0,16
Google Video Bot Disallow	17	0,15
ByteDance Bytespider Disallow	16	0,77
360 Bot Disallow	13	0,11
Google Favicon Disallow	12	0,1
Google API Bot Disallow	12	0,1
Google DuplexWeb Disallow	12	0,1
Webz Omgili Disallow	10	0,48
BuiltWithBot Disallow	8	0,07
Youdao Bot Disallow	6	0,05
Diffbot Disallow	4	0,19
Pipl Disallow	3	0,03
YOU Disallow	2	0,1