Robots.txt Distribuzione dell'utilizzo In Messico

Distribuzione per i siti Web che utilizzano Robots.txt tecnologie

Tecnologia	Siti web	%
Google AdsBot Disallow	56.813	18,47
Googlebot Disallow	42.035	13,66
Google AdsBot Mobile Disallow	36.308	11,8
Ahrefs Bot Disallow	22.586	7,34
Anthropic Claude Bot Disallow	10.783	3,51
Baidu Bot Disallow	10.770	3,5
Robots Disallow	9.292	3,02
GPTBot Disallow	9.048	2,94
Common Crawl Bot Disallow	7.414	2,41
Yandex Bot Disallow	6.924	2,25
Google Bard Disallow	6.907	2,25
Microsoft BingBot Disallow	6.859	2,23
Facebook Bot Disallow	6.841	2,22
AppleBot Disallow	6.821	2,22
Google AdsBot Mobile Apps Disallow	6.664	2,17
Apple AI Bot Disallow	6.656	2,16
Yahoo Slurp Disallow	6.590	2,14
Perplexity	6.500	2,11
Alexa Bot Disallow	6.459	2,1
Cohere AI Disallow	6.224	2,02
Sogou Bot Disallow	6.040	1,96
Some Robots Disallow	5.952	1,93
Soso Bot Disallow	5.924	1,93
Semrush Bot Disallow	3.015	0,98
Majestic Bot Disallow	1.797	0,58
Google Image Bot Disallow	1.262	0,41
Google Media Partners Disallow	1.066 ^*	0,35
Microsoft MSNBot Disallow	1.051	0,34
Webz Omgili Disallow	723	0,24
ByteDance Bytespider Disallow	676	0,22
Exalead Bot Disallow	432	0,14
Google News Bot Disallow	198	0,06
360 Bot Disallow	182	0,06
Google Video Bot Disallow	160	0,05
Google DuplexWeb Disallow	110	0,04
DuckDuckGo Bot Disallow	99	0,03
Youdao Bot Disallow	87	0,03
Google Favicon Disallow	73	0,02
Google API Bot Disallow	73	0,02
BuiltWithBot Disallow	67	0,02
Diffbot Disallow	63	0,02
YOU Disallow	43	0,01
Pipl Disallow	43	0,01
Google FeedFetcher Disallow	2	0
Google Read Aloud Disallow	1	0