Robots.txt Répartition de l'utilisation dans Estonie

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Googlebot Disallow	4 088	12,27
Majestic Bot Disallow	4 063	12,19
Google AdsBot Disallow	3 413	10,24
Robots Disallow	3 157	9,47
Google AdsBot Mobile Disallow	2 637	7,91
Baidu Bot Disallow	1 939	5,82
Ahrefs Bot Disallow	1 584	4,75
Some Robots Disallow	1 366	4,1
Yandex Bot Disallow	1 191	3,57
Alexa Bot Disallow	870	18,16
Semrush Bot Disallow	818	2,45
GPTBot Disallow	773	16,13
Google AdsBot Mobile Apps Disallow	689	2,07
Common Crawl Bot Disallow	660	13,77
Microsoft BingBot Disallow	646	1,94
Yahoo Slurp Disallow	588	1,76
Sogou Bot Disallow	535	1,61
Soso Bot Disallow	518	1,55
Anthropic Claude Bot Disallow	472	9,85
Google Bard Disallow	455	9,49
Facebook Bot Disallow	430	8,97
Cohere AI Disallow	406	8,47
AppleBot Disallow	292	0,88
Apple AI Bot Disallow	269	5,61
Exalead Bot Disallow	268	0,8
Perplexity	254	5,3
Google Media Partners Disallow	194 ^*	0,58
Google Image Bot Disallow	147	0,44
Microsoft MSNBot Disallow	128	0,38
Google News Bot Disallow	68	0,2
Google Video Bot Disallow	66	0,2
Google Favicon Disallow	55	0,17
DuckDuckGo Bot Disallow	50	0,15
Google API Bot Disallow	48	0,14
Google DuplexWeb Disallow	45	0,14
ByteDance Bytespider Disallow	37	0,77
Webz Omgili Disallow	34	0,71
360 Bot Disallow	23	0,07
BuiltWithBot Disallow	14	0,04
Diffbot Disallow	11	0,23
Youdao Bot Disallow	7	0,02
YOU Disallow	6	0,13
Pipl Disallow	5	0,02
Google Read Aloud Disallow	3	0,01