Robots.txt Distribución de uso en San Pedro y Miquelón

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Robots Disallow	298	19,35
Googlebot Disallow	263	17,08
Some Robots Disallow	115	7,47
Microsoft BingBot Disallow	105	6,82
Yandex Bot Disallow	101	6,56
Alexa Bot Disallow	100	39,53
Yahoo Slurp Disallow	99	6,43
Sogou Bot Disallow	95	6,17
Soso Bot Disallow	94	6,1
GPTBot Disallow	22	8,7
ByteDance Bytespider Disallow	21	8,3
Ahrefs Bot Disallow	16	1,04
Google Bard Disallow	16	6,32
Common Crawl Bot Disallow	15	5,93
Anthropic Claude Bot Disallow	15	5,93
Facebook Bot Disallow	15	5,93
AppleBot Disallow	14	0,91
Semrush Bot Disallow	14	0,91
Apple AI Bot Disallow	13	5,14
Majestic Bot Disallow	12	0,78
Google AdsBot Mobile Apps Disallow	11	0,71
Google Image Bot Disallow	9	0,58
Webz Omgili Disallow	9	3,56
Cohere AI Disallow	8	3,16
Perplexity	8	3,16
Microsoft MSNBot Disallow	7	0,45
Google News Bot Disallow	6	0,39
Google Video Bot Disallow	6	0,39
DuckDuckGo Bot Disallow	6	0,39
Google Favicon Disallow	6	0,39
Google API Bot Disallow	6	0,39
Google DuplexWeb Disallow	6	0,39
Exalead Bot Disallow	3	0,19
Diffbot Disallow	2	0,79
360 Bot Disallow	1	0,06
BuiltWithBot Disallow	1	0,06
Youdao Bot Disallow	1	0,06
YOU Disallow	1	0,4