Robots.txt Nutzungsverteilung in Vietnam

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Google AdsBot Disallow	8.024	10,97
Ahrefs Bot Disallow	7.980	10,91
Googlebot Disallow	7.811	10,68
Robots Disallow	5.652	7,73
Baidu Bot Disallow	4.468	6,11
Sogou Bot Disallow	4.085	5,58
Alexa Bot Disallow	3.142	4,3
Yandex Bot Disallow	3.079	4,21
Yahoo Slurp Disallow	2.877	3,93
Soso Bot Disallow	2.625	3,59
Some Robots Disallow	2.424	3,31
Majestic Bot Disallow	2.385	3,26
Microsoft BingBot Disallow	2.234	3,05
Semrush Bot Disallow	2.208	3,02
Google AdsBot Mobile Disallow	1.833	2,51
Google Media Partners Disallow	1.539 ^*	2,1
Exalead Bot Disallow	1.326	1,81
Common Crawl Bot Disallow	1.239	1,69
Microsoft MSNBot Disallow	1.013	1,38
360 Bot Disallow	929	1,27
GPTBot Disallow	827	1,13
Youdao Bot Disallow	760	1,04
YOU Disallow	755	1,03
Google Bard Disallow	545	0,75
Anthropic Claude Bot Disallow	444	0,61
AppleBot Disallow	443	0,61
Google Image Bot Disallow	411	0,56
Facebook Bot Disallow	392	0,54
Apple AI Bot Disallow	317	0,43
Cohere AI Disallow	299	0,41
Perplexity	287	0,39
Google AdsBot Mobile Apps Disallow	283	0,39
ByteDance Bytespider Disallow	155	0,21
Webz Omgili Disallow	84	0,11
Diffbot Disallow	52	0,07
Google Video Bot Disallow	44	0,06
Google News Bot Disallow	39	0,05
DuckDuckGo Bot Disallow	36	0,05
Google Favicon Disallow	28	0,04
BuiltWithBot Disallow	24	0,03
Google API Bot Disallow	21	0,03
Google DuplexWeb Disallow	21	0,03
Pipl Disallow	8	0,01
LLMS Text	6	0,01