Robots.txt Nutzungsverteilung In Andorra

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Googlebot Disallow	217	13,79
Google AdsBot Disallow	193	12,26
Google AdsBot Mobile Disallow	164	10,42
Some Robots Disallow	103	6,54
Robots Disallow	93	5,91
Ahrefs Bot Disallow	65	4,13
Baidu Bot Disallow	61	3,88
Yandex Bot Disallow	51	3,24
Alexa Bot Disallow	47	2,99
Yahoo Slurp Disallow	39	2,48
GPTBot Disallow	37	2,35
Common Crawl Bot Disallow	37	2,35
Sogou Bot Disallow	36	2,29
Anthropic Claude Bot Disallow	33	2,1
Soso Bot Disallow	32	2,03
Google AdsBot Mobile Apps Disallow	32	2,03
Majestic Bot Disallow	32	2,03
AppleBot Disallow	31	1,97
Microsoft BingBot Disallow	30	1,91
Google Bard Disallow	30	1,91
Facebook Bot Disallow	30	1,91
Apple AI Bot Disallow	29	1,84
Cohere AI Disallow	29	1,84
Perplexity	29	1,84
Semrush Bot Disallow	27	1,72
Google Media Partners Disallow	16 ^*	1,02
Microsoft MSNBot Disallow	11	0,7
Exalead Bot Disallow	9	0,57
ByteDance Bytespider Disallow	8	0,51
Google Image Bot Disallow	5	0,32
360 Bot Disallow	3	0,19
BuiltWithBot Disallow	3	0,19
Youdao Bot Disallow	3	0,19
YOU Disallow	2	0,13
Webz Omgili Disallow	1	0,06
Google News Bot Disallow	1	0,06
Google Video Bot Disallow	1	0,06
DuckDuckGo Bot Disallow	1	0,06
Google Favicon Disallow	1	0,06
Google API Bot Disallow	1	0,06
Google DuplexWeb Disallow	1	0,06