Robots.txt Répartition de l'utilisation dans France

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Googlebot Disallow	199 554	14,15
Google AdsBot Disallow	197 956	14,03
Google AdsBot Mobile Disallow	148 264	10,51
Robots Disallow	91 284	6,47
Ahrefs Bot Disallow	72 921	5,17
Baidu Bot Disallow	56 374	4
Sogou Bot Disallow	45 193	3,2
Yandex Bot Disallow	42 556	3,02
Microsoft BingBot Disallow	36 333	2,58
Yahoo Slurp Disallow	34 537	2,45
Some Robots Disallow	32 850	2,33
Soso Bot Disallow	31 156	2,21
Semrush Bot Disallow	30 200	2,14
Alexa Bot Disallow	29 936	2,12
GPTBot Disallow	29 541	2,09
Majestic Bot Disallow	27 825	1,97
Common Crawl Bot Disallow	26 266	1,86
Anthropic Claude Bot Disallow	25 023	1,77
Google Bard Disallow	22 794	1,62
Google AdsBot Mobile Apps Disallow	22 744	1,61
AppleBot Disallow	22 607	1,6
Facebook Bot Disallow	22 261	1,58
Perplexity	20 322	1,44
Apple AI Bot Disallow	20 238	1,43
Cohere AI Disallow	18 755	1,33
Google Media Partners Disallow	14 931 ^*	1,06
Google Image Bot Disallow	12 211	0,87
Microsoft MSNBot Disallow	9 747	0,69
DuckDuckGo Bot Disallow	7 303	0,52
ByteDance Bytespider Disallow	7 086	0,5
Youdao Bot Disallow	6 813	0,48
Exalead Bot Disallow	6 211	0,44
Google News Bot Disallow	5 847	0,41
Google Video Bot Disallow	5 810	0,41
Webz Omgili Disallow	5 652	0,4
Google Favicon Disallow	4 821	0,34
Google API Bot Disallow	4 769	0,34
Google DuplexWeb Disallow	4 749	0,34
360 Bot Disallow	2 371	0,17
Diffbot Disallow	1 924	0,14
YOU Disallow	1 703	0,12
BuiltWithBot Disallow	689	0,05
Pipl Disallow	514	0,04
Google FeedFetcher Disallow	28	0
Google Read Aloud Disallow	14	0
Google Web Light Disallow	4	0
LLMS Text	1	0