Robots.txt Distribución de uso en Dinamarca

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Google AdsBot Disallow	33.749	11,34
Ahrefs Bot Disallow	26.783	9
Googlebot Disallow	24.198	8,13
Google AdsBot Mobile Disallow	21.300	7,16
Majestic Bot Disallow	16.589	5,57
GPTBot Disallow	11.425	13,8
Microsoft BingBot Disallow	9.979	3,35
Common Crawl Bot Disallow	9.914	11,97
Yahoo Slurp Disallow	9.539	3,21
Robots Disallow	9.374	3,15
Anthropic Claude Bot Disallow	9.269	11,19
Google Bard Disallow	9.107	11
Google AdsBot Mobile Apps Disallow	8.921	3
Facebook Bot Disallow	8.854	10,69
AppleBot Disallow	8.687	2,92
Cohere AI Disallow	8.634	10,43
Perplexity	8.401	10,14
Apple AI Bot Disallow	8.212	9,92
Yandex Bot Disallow	6.714	2,26
Semrush Bot Disallow	6.660	2,24
Baidu Bot Disallow	6.588	2,21
Google Image Bot Disallow	6.266	2,11
Some Robots Disallow	6.178	2,08
Alexa Bot Disallow	5.269	6,36
Sogou Bot Disallow	3.963	1,33
Soso Bot Disallow	3.499	1,18
Exalead Bot Disallow	2.029	0,68
Microsoft MSNBot Disallow	1.017	0,34
ByteDance Bytespider Disallow	966	1,17
Webz Omgili Disallow	945	1,14
360 Bot Disallow	721	0,24
Google Media Partners Disallow	549 ^*	0,18
DuckDuckGo Bot Disallow	535	0,18
Diffbot Disallow	416	0,5
YOU Disallow	395	0,48
Google News Bot Disallow	393	0,13
Google Video Bot Disallow	389	0,13
Google Favicon Disallow	345	0,12
Google API Bot Disallow	310	0,1
Google DuplexWeb Disallow	292	0,1
BuiltWithBot Disallow	137	0,05
Pipl Disallow	38	0,01
Youdao Bot Disallow	12	0
Google FeedFetcher Disallow	2	0