Robots.txt Répartition de l'utilisation dans Russie

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Robots Disallow	680 036	16,91
Ahrefs Bot Disallow	477 099	11,86
Majestic Bot Disallow	471 658	11,73
Semrush Bot Disallow	446 283	11,1
Yandex Bot Disallow	342 330	8,51
Baidu Bot Disallow	300 643	7,48
Microsoft BingBot Disallow	284 995	7,09
AppleBot Disallow	279 702	6,95
Googlebot Disallow	176 747	4,39
Some Robots Disallow	129 563	3,22
Alexa Bot Disallow	109 718	22,25
Common Crawl Bot Disallow	52 869	10,72
Microsoft MSNBot Disallow	47 996	1,19
Exalead Bot Disallow	47 266	1,18
Yahoo Slurp Disallow	24 059	0,6
Sogou Bot Disallow	19 836	0,49
Soso Bot Disallow	19 351	0,48
GPTBot Disallow	16 010	3,25
Google AdsBot Disallow	15 866	0,39
Google AdsBot Mobile Disallow	14 102	0,35
Anthropic Claude Bot Disallow	9 144	1,85
Google Media Partners Disallow	8 853 ^*	0,22
Google Image Bot Disallow	7 631	0,19
Webz Omgili Disallow	7 516	1,52
Google Bard Disallow	7 082	1,44
Facebook Bot Disallow	6 401	1,3
360 Bot Disallow	4 094	0,1
BuiltWithBot Disallow	3 210	0,08
ByteDance Bytespider Disallow	2 956	0,6
Cohere AI Disallow	2 263	0,46
Google News Bot Disallow	1 029	0,03
Google Video Bot Disallow	919	0,02
Apple AI Bot Disallow	880	0,18
Google AdsBot Mobile Apps Disallow	851	0,02
Perplexity	599	0,12
DuckDuckGo Bot Disallow	464	0,01
Youdao Bot Disallow	379	0,01
Google API Bot Disallow	306	0,01
Google Favicon Disallow	305	0,01
Google DuplexWeb Disallow	283	0,01
YOU Disallow	229	0,05
Diffbot Disallow	126	0,03
Google FeedFetcher Disallow	111	0
Pipl Disallow	86	0
Google Read Aloud Disallow	1	0
Google BIG Bench Disallow	1	0