Robot.txt Distribuzione dell'utilizzo nel Bahrein

Distribuzione per siti web che utilizzano Robot.txt tecnologie

Tecnologia	Siti web	%
Google AdsBot Disallow	354	13,22
Googlebot Disallow	283	10,57
Google AdsBot Mobile Disallow	247	9,22
Robots Disallow	206	7,69
Ahrefs Bot Disallow	134	5
Baidu Bot Disallow	95	3,55
Common Crawl Bot Disallow	88	12,04
GPTBot Disallow	84	11,49
Sogou Bot Disallow	82	3,06
Anthropic Claude Bot Disallow	82	11,22
Google Bard Disallow	80	10,94
AppleBot Disallow	79	2,95
Facebook Bot Disallow	79	10,81
Apple AI Bot Disallow	77	10,53
Perplexity	76	10,4
Cohere AI Disallow	75	10,26
Google AdsBot Mobile Apps Disallow	71	2,65
Yandex Bot Disallow	70	2,61
Some Robots Disallow	65	2,43
Alexa Bot Disallow	58	7,93
Soso Bot Disallow	58	2,17
Yahoo Slurp Disallow	57	2,13
Microsoft BingBot Disallow	56	2,09
Majestic Bot Disallow	31	1,16
Semrush Bot Disallow	22	0,82
Google Image Bot Disallow	10	0,37
ByteDance Bytespider Disallow	9	1,23
Exalead Bot Disallow	7	0,26
Microsoft MSNBot Disallow	7	0,26
Webz Omgili Disallow	7	0,96
360 Bot Disallow	6	0,22
BuiltWithBot Disallow	6	0,22
Diffbot Disallow	4	0,55
YOU Disallow	4	0,55
Google News Bot Disallow	2	0,07
Google Video Bot Disallow	2	0,07
DuckDuckGo Bot Disallow	1	0,04
Google Favicon Disallow	1	0,04
Google API Bot Disallow	1	0,04
Google DuplexWeb Disallow	1	0,04
Pipl Disallow	1	0,04