Robots.txt Nutzungsverteilung In Malawi

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Googlebot Disallow	95	15,1
Robots Disallow	74	11,76
Google Media Partners Disallow	34 ^*	5,41
Common Crawl Bot Disallow	31	4,93
GPTBot Disallow	28	4,45
AppleBot Disallow	27	4,29
Anthropic Claude Bot Disallow	26	4,13
Google Bard Disallow	26	4,13
Facebook Bot Disallow	26	4,13
Apple AI Bot Disallow	26	4,13
Cohere AI Disallow	25	3,97
Perplexity	25	3,97
Google AdsBot Mobile Apps Disallow	25	3,97
Yandex Bot Disallow	21	3,34
Alexa Bot Disallow	18	2,86
Some Robots Disallow	17	2,7
Microsoft BingBot Disallow	17	2,7
Yahoo Slurp Disallow	17	2,7
Soso Bot Disallow	17	2,7
Ahrefs Bot Disallow	15	2,38
Semrush Bot Disallow	9	1,43
Majestic Bot Disallow	8	1,27
ByteDance Bytespider Disallow	3	0,48
Exalead Bot Disallow	3	0,48
Webz Omgili Disallow	2	0,32
360 Bot Disallow	2	0,32
Diffbot Disallow	2	0,32
YOU Disallow	2	0,32
Microsoft MSNBot Disallow	1	0,16
Google Image Bot Disallow	1	0,16
Google News Bot Disallow	1	0,16
Google Video Bot Disallow	1	0,16
DuckDuckGo Bot Disallow	1	0,16
Google Favicon Disallow	1	0,16
Google API Bot Disallow	1	0,16
Google DuplexWeb Disallow	1	0,16