Robots.txt Répartition de l'utilisation dans Corée du Sud

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Googlebot Disallow	39 873	15,93
Google AdsBot Disallow	23 251	9,29
Google AdsBot Mobile Disallow	22 024	8,8
Robots Disallow	20 730	8,28
Some Robots Disallow	20 452	8,17
Baidu Bot Disallow	10 841	4,33
Microsoft BingBot Disallow	9 501	3,8
Yandex Bot Disallow	8 223	3,28
Yahoo Slurp Disallow	6 946	2,77
Alexa Bot Disallow	6 697	2,68
Sogou Bot Disallow	6 628	2,65
Ahrefs Bot Disallow	6 404	2,56
Soso Bot Disallow	6 170	2,46
Majestic Bot Disallow	6 109	2,44
Common Crawl Bot Disallow	5 802	2,32
Semrush Bot Disallow	5 455	2,18
GPTBot Disallow	5 373	2,15
Anthropic Claude Bot Disallow	4 671	1,87
Facebook Bot Disallow	4 062	1,62
Google Bard Disallow	4 003	1,6
Cohere AI Disallow	3 767	1,5
ByteDance Bytespider Disallow	3 659	1,46
Webz Omgili Disallow	3 062	1,22
Diffbot Disallow	2 816	1,12
AppleBot Disallow	2 373	0,95
Microsoft MSNBot Disallow	2 224	0,89
Google Image Bot Disallow	1 801	0,72
Apple AI Bot Disallow	1 225	0,49
Perplexity	1 067	0,43
Google AdsBot Mobile Apps Disallow	1 053	0,42
Exalead Bot Disallow	1 012	0,4
Google Media Partners Disallow	792 ^*	0,32
360 Bot Disallow	770	0,31
DuckDuckGo Bot Disallow	645	0,26
Pipl Disallow	256	0,1
BuiltWithBot Disallow	248	0,1
YOU Disallow	104	0,04
Youdao Bot Disallow	65	0,03
Google News Bot Disallow	51	0,02
Google Video Bot Disallow	41	0,02
Google API Bot Disallow	40	0,02
Google DuplexWeb Disallow	22	0,01
Google Favicon Disallow	21	0,01
Google FeedFetcher Disallow	2	0
Google Read Aloud Disallow	2	0
Google Web Light Disallow	1	0