Robots.txt Distribución de uso en Katar

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Google AdsBot Disallow	1.085	15,3
Googlebot Disallow	841	11,86
Google AdsBot Mobile Disallow	651	9,18
Ahrefs Bot Disallow	505	7,12
Robots Disallow	372	5,24
Baidu Bot Disallow	274	3,86
Yandex Bot Disallow	252	3,55
Alexa Bot Disallow	224	3,16
Yahoo Slurp Disallow	223	3,14
Microsoft BingBot Disallow	220	3,1
Sogou Bot Disallow	217	3,06
Soso Bot Disallow	202	2,85
Common Crawl Bot Disallow	170	2,4
GPTBot Disallow	167	2,35
Google Bard Disallow	156	2,2
AppleBot Disallow	155	2,19
Anthropic Claude Bot Disallow	155	2,19
Facebook Bot Disallow	152	2,14
Apple AI Bot Disallow	149	2,1
Perplexity	149	2,1
Cohere AI Disallow	142	2
Google AdsBot Mobile Apps Disallow	138	1,95
Some Robots Disallow	132	1,86
Google Media Partners Disallow	97 ^*	1,37
Majestic Bot Disallow	88	1,24
Semrush Bot Disallow	65	0,92
Microsoft MSNBot Disallow	28	0,39
ByteDance Bytespider Disallow	20	0,28
Exalead Bot Disallow	17	0,24
Webz Omgili Disallow	12	0,17
Google Image Bot Disallow	10	0,14
360 Bot Disallow	6	0,08
BuiltWithBot Disallow	5	0,07
Diffbot Disallow	4	0,06
Google News Bot Disallow	2	0,03
Google Video Bot Disallow	2	0,03
YOU Disallow	2	0,03
DuckDuckGo Bot Disallow	1	0,01
Google Favicon Disallow	1	0,01
Google API Bot Disallow	1	0,01
Pipl Disallow	1	0,01