Robots.txt Nutzungsverteilung In Vietnam

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Google AdsBot Disallow	13.253	13,68
Ahrefs Bot Disallow	12.597	13
Googlebot Disallow	10.024	10,34
Robots Disallow	6.398	6,6
Baidu Bot Disallow	5.440	5,61
Alexa Bot Disallow	3.982	37,09
Yandex Bot Disallow	3.946	4,07
Majestic Bot Disallow	3.882	4,01
Yahoo Slurp Disallow	3.693	3,81
Some Robots Disallow	3.567	3,68
Sogou Bot Disallow	3.540	3,65
Soso Bot Disallow	3.279	3,38
Semrush Bot Disallow	3.221	3,32
Microsoft BingBot Disallow	2.748	2,84
Google AdsBot Mobile Disallow	2.626	2,71
Exalead Bot Disallow	1.863	1,92
Common Crawl Bot Disallow	1.646	15,33
Microsoft MSNBot Disallow	1.439	1,49
360 Bot Disallow	1.362	1,41
Youdao Bot Disallow	1.014	1,05
GPTBot Disallow	915	8,52
YOU Disallow	819	7,63
Google Media Partners Disallow	792 ^*	0,82
Google Bard Disallow	606	5,64
Anthropic Claude Bot Disallow	560	5,22
AppleBot Disallow	528	0,54
Google Image Bot Disallow	528	0,54
Facebook Bot Disallow	424	3,95
Google AdsBot Mobile Apps Disallow	415	0,43
Apple AI Bot Disallow	386	3,6
Cohere AI Disallow	367	3,42
Perplexity	359	3,34
ByteDance Bytespider Disallow	229	2,13
Webz Omgili Disallow	77	0,72
Google Video Bot Disallow	59	0,06
Google News Bot Disallow	58	0,06
Diffbot Disallow	55	0,51
DuckDuckGo Bot Disallow	54	0,06
BuiltWithBot Disallow	39	0,04
Google Favicon Disallow	37	0,04
Google API Bot Disallow	30	0,03
Google DuplexWeb Disallow	30	0,03
Pipl Disallow	11	0,01
Google FeedFetcher Disallow	1	0