Robots.txt Nutzungsverteilung in Ålandinseln

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Google AdsBot Disallow	55	20,52
GPTBot Disallow	16	5,97
AppleBot Disallow	16	5,97
Common Crawl Bot Disallow	15	5,6
Anthropic Claude Bot Disallow	15	5,6
Google Bard Disallow	15	5,6
Facebook Bot Disallow	15	5,6
Apple AI Bot Disallow	15	5,6
Perplexity	15	5,6
Cohere AI Disallow	14	5,22
Robots Disallow	13	4,85
Google AdsBot Mobile Apps Disallow	11	4,1
Ahrefs Bot Disallow	6	2,24
ByteDance Bytespider Disallow	5	1,87
Webz Omgili Disallow	5	1,87
Yandex Bot Disallow	4	1,49
Diffbot Disallow	4	1,49
YOU Disallow	4	1,49
Microsoft BingBot Disallow	3	1,12
Yahoo Slurp Disallow	3	1,12
Majestic Bot Disallow	3	1,12
Semrush Bot Disallow	3	1,12
Alexa Bot Disallow	2	0,75
Soso Bot Disallow	2	0,75
Microsoft MSNBot Disallow	2	0,75
Google Image Bot Disallow	1	0,37
Google News Bot Disallow	1	0,37
Google Video Bot Disallow	1	0,37
DuckDuckGo Bot Disallow	1	0,37
Google Favicon Disallow	1	0,37
Google API Bot Disallow	1	0,37
Google DuplexWeb Disallow	1	0,37