Robots.txt Nutzungsverteilung In Senegal

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Googlebot Disallow	229	18,07
Robots Disallow	199	15,71
Ahrefs Bot Disallow	60	4,74
GPTBot Disallow	59	14,08
Common Crawl Bot Disallow	57	13,6
Google AdsBot Mobile Apps Disallow	50	3,95
Yandex Bot Disallow	48	3,79
Google Bard Disallow	46	10,98
AppleBot Disallow	45	3,55
Anthropic Claude Bot Disallow	45	10,74
Facebook Bot Disallow	44	10,5
Apple AI Bot Disallow	44	10,5
Cohere AI Disallow	44	10,5
Perplexity	44	10,5
Microsoft BingBot Disallow	36	2,84
Alexa Bot Disallow	34	8,11
Soso Bot Disallow	34	2,68
Sogou Bot Disallow	32	2,53
Yahoo Slurp Disallow	31	2,45
Majestic Bot Disallow	26	2,05
Semrush Bot Disallow	22	1,74
Exalead Bot Disallow	9	0,71
360 Bot Disallow	6	0,47
Microsoft MSNBot Disallow	5	0,39
Google Image Bot Disallow	5	0,39
BuiltWithBot Disallow	4	0,32
Google News Bot Disallow	3	0,24
DuckDuckGo Bot Disallow	2	0,16
Webz Omgili Disallow	1	0,24
Youdao Bot Disallow	1	0,08
YOU Disallow	1	0,24
Pipl Disallow	1	0,08