Robots.txt Répartition des utilisations au Canada

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Google AdsBot Disallow	296 625	15,57
Google AdsBot Mobile Disallow	224 078	11,76
Googlebot Disallow	190 473	10
GPTBot Disallow	77 999	4,09
Common Crawl Bot Disallow	75 989	3,99
Anthropic Claude Bot Disallow	74 147	3,89
Google Bard Disallow	73 388	3,85
Facebook Bot Disallow	73 228	3,84
AppleBot Disallow	73 223	3,84
Apple AI Bot Disallow	71 817	3,77
Cohere AI Disallow	71 666	3,76
Perplexity	71 562	3,76
Google AdsBot Mobile Apps Disallow	69 600	3,65
Ahrefs Bot Disallow	68 605	3,6
Baidu Bot Disallow	56 474	2,96
Sogou Bot Disallow	43 134	2,26
Robots Disallow	39 430	2,07
Yandex Bot Disallow	36 600	1,92
Some Robots Disallow	33 292	1,75
Microsoft BingBot Disallow	29 762	1,56
Alexa Bot Disallow	28 998	1,52
Yahoo Slurp Disallow	28 707	1,51
Soso Bot Disallow	25 944	1,36
Semrush Bot Disallow	15 690	0,82
Majestic Bot Disallow	15 445	0,81
ByteDance Bytespider Disallow	5 784	0,3
Google Media Partners Disallow	5 050 ^*	0,27
Microsoft MSNBot Disallow	5 008	0,26
Google Image Bot Disallow	4 250	0,22
Webz Omgili Disallow	3 795	0,2
Exalead Bot Disallow	3 279	0,17
Diffbot Disallow	1 813	0,1
360 Bot Disallow	1 701	0,09
Google News Bot Disallow	1 287	0,07
Google Video Bot Disallow	1 159	0,06
Google Favicon Disallow	1 114	0,06
BuiltWithBot Disallow	891	0,05
YOU Disallow	830	0,04
DuckDuckGo Bot Disallow	680	0,04
Google API Bot Disallow	643	0,03
Google DuplexWeb Disallow	632	0,03
Pipl Disallow	406	0,02
Google FeedFetcher Disallow	387	0,02
Google Read Aloud Disallow	374	0,02
Google Web Light Disallow	371	0,02
Youdao Bot Disallow	243	0,01