Robots.txt Répartition de l'utilisation dans Bahreïn

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Google AdsBot Disallow	483	14,73
Googlebot Disallow	359	10,95
Google AdsBot Mobile Disallow	341	10,4
Ahrefs Bot Disallow	166	5,06
Robots Disallow	138	4,21
Common Crawl Bot Disallow	117	3,57
GPTBot Disallow	113	3,45
Google Bard Disallow	109	3,32
Anthropic Claude Bot Disallow	108	3,29
Facebook Bot Disallow	107	3,26
AppleBot Disallow	105	3,2
Apple AI Bot Disallow	104	3,17
Cohere AI Disallow	103	3,14
Perplexity	103	3,14
Google AdsBot Mobile Apps Disallow	98	2,99
Baidu Bot Disallow	94	2,87
Yandex Bot Disallow	76	2,32
Sogou Bot Disallow	71	2,17
Soso Bot Disallow	71	2,17
Yahoo Slurp Disallow	68	2,07
Alexa Bot Disallow	67	2,04
Microsoft BingBot Disallow	66	2,01
Some Robots Disallow	58	1,77
Majestic Bot Disallow	35	1,07
Google Media Partners Disallow	32 ^*	0,98
Semrush Bot Disallow	18	0,55
Exalead Bot Disallow	10	0,3
Google Image Bot Disallow	10	0,3
360 Bot Disallow	9	0,27
BuiltWithBot Disallow	9	0,27
ByteDance Bytespider Disallow	8	0,24
Microsoft MSNBot Disallow	8	0,24
Webz Omgili Disallow	6	0,18
Diffbot Disallow	4	0,12
YOU Disallow	3	0,09
Youdao Bot Disallow	1	0,03
Pipl Disallow	1	0,03