Robots.txt Nutzungsverteilung In Taiwan

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Googlebot Disallow	15.420	10,37
Baidu Bot Disallow	12.017	8,09
Yandex Bot Disallow	10.770	7,25
Robots Disallow	10.342	6,96
Sogou Bot Disallow	9.561	6,43
Microsoft BingBot Disallow	8.965	6,03
Yahoo Slurp Disallow	8.899	5,99
Alexa Bot Disallow	8.692	5,85
Soso Bot Disallow	8.642	5,81
Google AdsBot Disallow	7.655	5,15
Ahrefs Bot Disallow	5.810	3,91
Majestic Bot Disallow	5.774	3,88
Semrush Bot Disallow	3.854	2,59
Google AdsBot Mobile Disallow	3.796	2,55
Anthropic Claude Bot Disallow	3.644	2,45
Some Robots Disallow	3.210	2,16
Common Crawl Bot Disallow	2.936	1,98
GPTBot Disallow	2.736	1,84
AppleBot Disallow	1.640	1,1
Google Image Bot Disallow	1.510	1,02
Google Bard Disallow	1.496	1,01
Facebook Bot Disallow	1.461	0,98
Apple AI Bot Disallow	1.429	0,96
Cohere AI Disallow	1.324	0,89
ByteDance Bytespider Disallow	1.169	0,79
Google Media Partners Disallow	1.086 ^*	0,73
Perplexity	828	0,56
Google AdsBot Mobile Apps Disallow	820	0,55
Webz Omgili Disallow	787	0,53
Diffbot Disallow	531	0,36
Microsoft MSNBot Disallow	446	0,3
360 Bot Disallow	371	0,25
Exalead Bot Disallow	278	0,19
Youdao Bot Disallow	109	0,07
DuckDuckGo Bot Disallow	105	0,07
Google Video Bot Disallow	103	0,07
Google News Bot Disallow	85	0,06
Google API Bot Disallow	80	0,05
Google Favicon Disallow	72	0,05
Google DuplexWeb Disallow	59	0,04
BuiltWithBot Disallow	52	0,03
YOU Disallow	51	0,03
Google FeedFetcher Disallow	8	0,01
Pipl Disallow	6	0
Google Read Aloud Disallow	1	0