Robots.txt Répartition de l'utilisation dans L'Autriche

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Googlebot Disallow	52 212	14,48
Google AdsBot Disallow	43 233	11,99
Google AdsBot Mobile Disallow	36 908	10,24
Baidu Bot Disallow	22 221	6,16
Robots Disallow	17 437	4,84
Ahrefs Bot Disallow	14 108	3,91
Some Robots Disallow	12 304	3,41
Yandex Bot Disallow	11 356	3,15
Microsoft BingBot Disallow	10 453	2,9
Yahoo Slurp Disallow	9 873	2,74
Alexa Bot Disallow	9 799	2,72
GPTBot Disallow	8 411	2,33
Sogou Bot Disallow	8 102	2,25
Soso Bot Disallow	8 028	2,23
Google AdsBot Mobile Apps Disallow	8 011	2,22
Common Crawl Bot Disallow	7 539	2,09
Anthropic Claude Bot Disallow	7 023	1,95
Google Bard Disallow	6 557	1,82
Facebook Bot Disallow	6 397	1,77
AppleBot Disallow	6 295	1,75
Apple AI Bot Disallow	6 069	1,68
Cohere AI Disallow	6 066	1,68
Majestic Bot Disallow	6 041	1,68
Perplexity	5 949	1,65
Semrush Bot Disallow	5 416	1,5
Google Media Partners Disallow	2 843 ^*	0,79
Google Image Bot Disallow	2 832	0,79
Microsoft MSNBot Disallow	2 510	0,7
ByteDance Bytespider Disallow	2 287	0,63
Google News Bot Disallow	2 001	0,56
Google Video Bot Disallow	1 937	0,54
DuckDuckGo Bot Disallow	1 924	0,53
Google Favicon Disallow	1 882	0,52
Google API Bot Disallow	1 852	0,51
Google DuplexWeb Disallow	1 849	0,51
Exalead Bot Disallow	1 083	0,3
360 Bot Disallow	567	0,16
Webz Omgili Disallow	559	0,16
BuiltWithBot Disallow	298	0,08
Diffbot Disallow	136	0,04
YOU Disallow	82	0,02
Pipl Disallow	39	0,01
Youdao Bot Disallow	18	0
Google FeedFetcher Disallow	2	0
Google Read Aloud Disallow	1	0
Google BIG Bench Disallow	1	0