Robots.txt Distribución de uso en bahamas

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Google AdsBot Disallow	298	8,85
Google AdsBot Mobile Disallow	256	7,6
Googlebot Disallow	199	5,91
GPTBot Disallow	188	5,58
Common Crawl Bot Disallow	180	5,35
Google AdsBot Mobile Apps Disallow	160	4,75
Google Bard Disallow	158	4,69
Anthropic Claude Bot Disallow	155	4,6
Facebook Bot Disallow	154	4,57
AppleBot Disallow	152	4,51
Perplexity	150	4,46
Cohere AI Disallow	146	4,34
Apple AI Bot Disallow	145	4,31
Robots Disallow	112	3,33
Ahrefs Bot Disallow	98	2,91
Baidu Bot Disallow	88	2,61
Microsoft BingBot Disallow	73	2,17
Yandex Bot Disallow	71	2,11
Yahoo Slurp Disallow	67	1,99
Sogou Bot Disallow	67	1,99
Soso Bot Disallow	67	1,99
Alexa Bot Disallow	64	1,9
Majestic Bot Disallow	50	1,49
Some Robots Disallow	42	1,25
Semrush Bot Disallow	40	1,19
ByteDance Bytespider Disallow	25	0,74
Exalead Bot Disallow	24	0,71
Microsoft MSNBot Disallow	19	0,56
360 Bot Disallow	19	0,56
BuiltWithBot Disallow	18	0,53
Google Media Partners Disallow	16 ^*	0,48
Google Image Bot Disallow	13	0,39
Webz Omgili Disallow	8	0,24
YOU Disallow	7	0,21
Google News Bot Disallow	6	0,18
Google Video Bot Disallow	6	0,18
Diffbot Disallow	6	0,18
DuckDuckGo Bot Disallow	5	0,15
Google Favicon Disallow	5	0,15
Google API Bot Disallow	4	0,12
Google DuplexWeb Disallow	4	0,12
Youdao Bot Disallow	2	0,06