Robots.txt Distribución de uso en Letonia

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Googlebot Disallow	4.420	13,04
Google AdsBot Disallow	4.149	12,24
Google AdsBot Mobile Disallow	2.859	8,43
Ahrefs Bot Disallow	1.957	5,77
Yandex Bot Disallow	1.786	5,27
Robots Disallow	1.757	5,18
Baidu Bot Disallow	1.655	4,88
Sogou Bot Disallow	1.475	4,35
Alexa Bot Disallow	1.307	3,86
Soso Bot Disallow	1.217	3,59
Microsoft BingBot Disallow	1.212	3,57
Yahoo Slurp Disallow	1.163	3,43
Some Robots Disallow	955	2,82
Majestic Bot Disallow	891	2,63
Semrush Bot Disallow	741	2,19
Common Crawl Bot Disallow	624	1,84
GPTBot Disallow	522	1,54
Anthropic Claude Bot Disallow	496	1,46
Google Bard Disallow	474	1,4
Google Media Partners Disallow	470 ^*	1,39
Facebook Bot Disallow	467	1,38
AppleBot Disallow	460	1,36
Apple AI Bot Disallow	426	1,26
Google AdsBot Mobile Apps Disallow	416	1,23
Cohere AI Disallow	408	1,2
Perplexity	403	1,19
Exalead Bot Disallow	245	0,72
ByteDance Bytespider Disallow	177	0,52
Microsoft MSNBot Disallow	128	0,38
360 Bot Disallow	118	0,35
Google Image Bot Disallow	102	0,3
BuiltWithBot Disallow	99	0,29
Webz Omgili Disallow	70	0,21
DuckDuckGo Bot Disallow	43	0,13
Google Video Bot Disallow	37	0,11
Google News Bot Disallow	35	0,1
Google Favicon Disallow	34	0,1
Google API Bot Disallow	33	0,1
Google DuplexWeb Disallow	33	0,1
Diffbot Disallow	21	0,06
YOU Disallow	8	0,02
Pipl Disallow	7	0,02
Youdao Bot Disallow	2	0,01
Google Read Aloud Disallow	1	0