Robots.txt Distribución de uso en Filipinas

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Some Robots Disallow	81.695	47,38
Googlebot Disallow	11.177	6,48
Baidu Bot Disallow	8.296	4,81
Sogou Bot Disallow	7.211	4,18
Google AdsBot Disallow	6.633	3,85
Yandex Bot Disallow	6.591	3,82
Alexa Bot Disallow	6.452	3,74
Microsoft BingBot Disallow	6.401	3,71
Yahoo Slurp Disallow	6.368	3,69
Soso Bot Disallow	6.302	3,66
Robots Disallow	4.683	2,72
Google AdsBot Mobile Disallow	4.020	2,33
Ahrefs Bot Disallow	2.935	1,7
GPTBot Disallow	1.181	0,68
Common Crawl Bot Disallow	1.172	0,68
Anthropic Claude Bot Disallow	1.125	0,65
Google Bard Disallow	1.120	0,65
AppleBot Disallow	1.118	0,65
Facebook Bot Disallow	1.093	0,63
Apple AI Bot Disallow	1.075	0,62
Perplexity	1.031	0,6
Cohere AI Disallow	1.015	0,59
Google AdsBot Mobile Apps Disallow	1.000	0,58
Google Media Partners Disallow	794 ^*	0,46
Semrush Bot Disallow	433	0,25
Majestic Bot Disallow	403	0,23
360 Bot Disallow	377	0,22
Microsoft MSNBot Disallow	119	0,07
Exalead Bot Disallow	113	0,07
Webz Omgili Disallow	111	0,06
ByteDance Bytespider Disallow	105	0,06
Google Image Bot Disallow	95	0,06
Diffbot Disallow	29	0,02
BuiltWithBot Disallow	25	0,01
DuckDuckGo Bot Disallow	21	0,01
Google News Bot Disallow	17	0,01
Google Video Bot Disallow	14	0,01
YOU Disallow	12	0,01
Google Favicon Disallow	11	0,01
Google API Bot Disallow	11	0,01
Youdao Bot Disallow	10	0,01
Pipl Disallow	10	0,01
Google DuplexWeb Disallow	9	0,01
Google FeedFetcher Disallow	3	0