Robots.txt Nutzungsverteilung In Grönland

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Robots Disallow	116	9,71
Googlebot Disallow	104	8,7
Google AdsBot Disallow	101	8,45
Google AdsBot Mobile Disallow	72	6,03
Ahrefs Bot Disallow	56	4,69
Baidu Bot Disallow	55	4,6
GPTBot Disallow	47	3,93
Some Robots Disallow	44	3,68
Common Crawl Bot Disallow	44	3,68
Anthropic Claude Bot Disallow	42	3,51
Google Bard Disallow	41	3,43
Facebook Bot Disallow	40	3,35
AppleBot Disallow	39	3,26
Apple AI Bot Disallow	37	3,1
Cohere AI Disallow	35	2,93
Perplexity	35	2,93
Yandex Bot Disallow	34	2,85
Google AdsBot Mobile Apps Disallow	33	2,76
Microsoft BingBot Disallow	32	2,68
Alexa Bot Disallow	30	2,51
Yahoo Slurp Disallow	29	2,43
Soso Bot Disallow	27	2,26
Majestic Bot Disallow	25	2,09
Google Media Partners Disallow	16 ^*	1,34
Webz Omgili Disallow	10	0,84
Semrush Bot Disallow	9	0,75
ByteDance Bytespider Disallow	9	0,75
Microsoft MSNBot Disallow	5	0,42
Google Image Bot Disallow	4	0,33
Diffbot Disallow	4	0,33
Exalead Bot Disallow	3	0,25
Google News Bot Disallow	2	0,17
Google Video Bot Disallow	2	0,17
YOU Disallow	2	0,17
DuckDuckGo Bot Disallow	2	0,17
Google Favicon Disallow	2	0,17
Google API Bot Disallow	2	0,17
Google DuplexWeb Disallow	2	0,17
360 Bot Disallow	1	0,08
BuiltWithBot Disallow	1	0,08
Pipl Disallow	1	0,08