Robots.txt Distribución de uso en Líbano

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Google AdsBot Disallow	1.270	14,62
Ahrefs Bot Disallow	735	8,46
GPTBot Disallow	506	13,93
Common Crawl Bot Disallow	501	13,79
Google AdsBot Mobile Apps Disallow	438	5,04
Anthropic Claude Bot Disallow	415	11,42
Google Bard Disallow	413	11,37
Facebook Bot Disallow	413	11,37
AppleBot Disallow	411	4,73
Cohere AI Disallow	407	11,2
Apple AI Bot Disallow	402	11,07
Perplexity	399	10,98
Googlebot Disallow	369	4,25
Robots Disallow	316	3,64
Some Robots Disallow	176	2,03
Baidu Bot Disallow	156	1,8
Yandex Bot Disallow	126	1,45
Semrush Bot Disallow	117	1,35
Sogou Bot Disallow	116	1,34
Microsoft BingBot Disallow	114	1,31
Soso Bot Disallow	113	1,3
Yahoo Slurp Disallow	109	1,25
Majestic Bot Disallow	104	1,2
Alexa Bot Disallow	91	2,5
Exalead Bot Disallow	66	0,76
360 Bot Disallow	61	0,7
Microsoft MSNBot Disallow	60	0,69
ByteDance Bytespider Disallow	59	1,62
BuiltWithBot Disallow	57	0,66
Google Image Bot Disallow	48	0,55
Google News Bot Disallow	38	0,44
Google Video Bot Disallow	37	0,43
Google Media Partners Disallow	32 ^*	0,37
Webz Omgili Disallow	6	0,17
Youdao Bot Disallow	5	0,06
YOU Disallow	1	0,03
Pipl Disallow	1	0,01