Robots.txt Distribución de uso en Tailandia

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Ahrefs Bot Disallow	12.399	10,39
Majestic Bot Disallow	11.981	10,04
Semrush Bot Disallow	11.921	9,99
Googlebot Disallow	11.680	9,79
Alexa Bot Disallow	11.632	49,92
Google AdsBot Disallow	9.441	7,91
Google AdsBot Mobile Disallow	8.289	6,94
Some Robots Disallow	5.413	4,54
Baidu Bot Disallow	5.035	4,22
Yandex Bot Disallow	4.092	3,43
Common Crawl Bot Disallow	3.029	13
Robots Disallow	2.642	2,21
Microsoft BingBot Disallow	2.205	1,85
Yahoo Slurp Disallow	2.151	1,8
Exalead Bot Disallow	1.823	1,53
Sogou Bot Disallow	1.701	1,43
Soso Bot Disallow	1.611	1,35
GPTBot Disallow	1.332	5,72
Anthropic Claude Bot Disallow	1.270	5,45
AppleBot Disallow	1.200	1,01
Google AdsBot Mobile Apps Disallow	1.124	0,94
Google Bard Disallow	1.089	4,67
Facebook Bot Disallow	1.072	4,6
Cohere AI Disallow	1.043	4,48
Apple AI Bot Disallow	1.033	4,43
Perplexity	1.026	4,4
Google Media Partners Disallow	646 ^*	0,54
DuckDuckGo Bot Disallow	426	0,36
Google Image Bot Disallow	232	0,19
Microsoft MSNBot Disallow	167	0,14
360 Bot Disallow	152	0,13
ByteDance Bytespider Disallow	129	0,55
BuiltWithBot Disallow	73	0,06
Webz Omgili Disallow	53	0,23
Google News Bot Disallow	52	0,04
Youdao Bot Disallow	47	0,04
Google Video Bot Disallow	42	0,04
Diffbot Disallow	23	0,1
Google Favicon Disallow	21	0,02
Google API Bot Disallow	17	0,01
Google DuplexWeb Disallow	16	0,01
Pipl Disallow	13	0,01
YOU Disallow	12	0,05
Google FeedFetcher Disallow	1	0