Robots.txt Répartition de l'utilisation dans Corée du Sud

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Googlebot Disallow	38 933	15,39
Google AdsBot Disallow	22 360	8,84
Google AdsBot Mobile Disallow	21 149	8,36
Some Robots Disallow	20 770	8,21
Robots Disallow	18 867	7,46
Baidu Bot Disallow	11 080	4,38
Microsoft BingBot Disallow	10 252	4,05
Sogou Bot Disallow	8 708	3,44
Yandex Bot Disallow	8 298	3,28
Yahoo Slurp Disallow	6 959	2,75
Alexa Bot Disallow	6 674	2,64
Majestic Bot Disallow	6 421	2,54
Ahrefs Bot Disallow	6 335	2,5
Soso Bot Disallow	6 071	2,4
Semrush Bot Disallow	5 857	2,32
GPTBot Disallow	5 832	2,31
Common Crawl Bot Disallow	5 813	2,3
Anthropic Claude Bot Disallow	4 908	1,94
Facebook Bot Disallow	4 285	1,69
Google Bard Disallow	4 236	1,67
Cohere AI Disallow	3 988	1,58
ByteDance Bytespider Disallow	3 975	1,57
Webz Omgili Disallow	3 249	1,28
Diffbot Disallow	2 964	1,17
AppleBot Disallow	2 628	1,04
Microsoft MSNBot Disallow	2 237	0,88
Google Image Bot Disallow	1 790	0,71
Google Media Partners Disallow	1 378 ^*	0,54
Apple AI Bot Disallow	1 344	0,53
Perplexity	1 172	0,46
Google AdsBot Mobile Apps Disallow	1 020	0,4
Exalead Bot Disallow	971	0,38
360 Bot Disallow	801	0,32
DuckDuckGo Bot Disallow	646	0,26
Pipl Disallow	322	0,13
BuiltWithBot Disallow	226	0,09
YOU Disallow	156	0,06
Youdao Bot Disallow	71	0,03
Google News Bot Disallow	48	0,02
Google API Bot Disallow	39	0,02
Google Video Bot Disallow	36	0,01
Google DuplexWeb Disallow	21	0,01
Google Favicon Disallow	20	0,01
Google FeedFetcher Disallow	2	0
Google Read Aloud Disallow	1	0
Google Web Light Disallow	1	0