Robots.txt Nutzungsverteilung In Kasachstan

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Yandex Bot Disallow	3.745	26,93
Robots Disallow	2.157	15,51
Googlebot Disallow	2.085	14,99
Semrush Bot Disallow	884	6,36
Majestic Bot Disallow	823	5,92
Ahrefs Bot Disallow	749	5,39
Microsoft MSNBot Disallow	409	2,94
Microsoft BingBot Disallow	352	2,53
Google AdsBot Disallow	299	2,15
Some Robots Disallow	292	2,1
GPTBot Disallow	279	2,01
Google AdsBot Mobile Disallow	236	1,7
Alexa Bot Disallow	225	1,62
Common Crawl Bot Disallow	191	1,37
Google Media Partners Disallow	178 ^*	1,28
Anthropic Claude Bot Disallow	139	1
Google Bard Disallow	126	0,91
Baidu Bot Disallow	121	0,87
Yahoo Slurp Disallow	112	0,81
Webz Omgili Disallow	106	0,76
Sogou Bot Disallow	74	0,53
Google Image Bot Disallow	64	0,46
Soso Bot Disallow	47	0,34
Exalead Bot Disallow	37	0,27
AppleBot Disallow	29	0,21
ByteDance Bytespider Disallow	25	0,18
Facebook Bot Disallow	18	0,13
Apple AI Bot Disallow	13	0,09
Cohere AI Disallow	12	0,09
Google AdsBot Mobile Apps Disallow	12	0,09
360 Bot Disallow	12	0,09
Perplexity	10	0,07
Google News Bot Disallow	10	0,07
Google Video Bot Disallow	6	0,04
DuckDuckGo Bot Disallow	6	0,04
Google Favicon Disallow	5	0,04
Google API Bot Disallow	5	0,04
Google DuplexWeb Disallow	5	0,04
Diffbot Disallow	3	0,02
BuiltWithBot Disallow	2	0,01
Youdao Bot Disallow	2	0,01
YOU Disallow	1	0,01
Pipl Disallow	1	0,01
Google FeedFetcher Disallow	1	0,01
Google Read Aloud Disallow	1	0,01