Robots.txt Répartition de l'utilisation dans Estonie

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Googlebot Disallow	4 142	11,9
Majestic Bot Disallow	4 022	11,56
Google AdsBot Disallow	3 401	9,77
Google AdsBot Mobile Disallow	2 629	7,56
Robots Disallow	2 563	7,37
Baidu Bot Disallow	2 039	5,86
Ahrefs Bot Disallow	1 600	4,6
Some Robots Disallow	1 481	4,26
Yandex Bot Disallow	1 081	3,11
Semrush Bot Disallow	832	2,39
GPTBot Disallow	810	12,19
Common Crawl Bot Disallow	793	11,93
Alexa Bot Disallow	769	11,57
Anthropic Claude Bot Disallow	715	10,76
Google AdsBot Mobile Apps Disallow	704	2,02
Google Bard Disallow	692	10,41
AppleBot Disallow	671	1,93
Facebook Bot Disallow	665	10
Apple AI Bot Disallow	644	9,69
Cohere AI Disallow	629	9,46
Perplexity	627	9,43
Microsoft BingBot Disallow	570	1,64
Yahoo Slurp Disallow	509	1,46
Sogou Bot Disallow	459	1,32
Soso Bot Disallow	446	1,28
Exalead Bot Disallow	279	0,8
Google Media Partners Disallow	194 ^*	0,56
Google Image Bot Disallow	154	0,44
Microsoft MSNBot Disallow	130	0,37
Google News Bot Disallow	68	0,2
Google Video Bot Disallow	66	0,19
ByteDance Bytespider Disallow	62	0,93
Google Favicon Disallow	57	0,16
Webz Omgili Disallow	55	0,83
DuckDuckGo Bot Disallow	51	0,15
Google API Bot Disallow	51	0,15
Google DuplexWeb Disallow	47	0,14
360 Bot Disallow	24	0,07
Diffbot Disallow	18	0,27
BuiltWithBot Disallow	15	0,04
YOU Disallow	12	0,18
Pipl Disallow	8	0,02
Youdao Bot Disallow	7	0,02
Google Read Aloud Disallow	3	0,01
Google FeedFetcher Disallow	1	0