Robots.txt Répartition de l'utilisation dans Somalie

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Googlebot Disallow	1 234	16,62
Baidu Bot Disallow	1 173	15,8
Some Robots Disallow	883	11,89
Sogou Bot Disallow	826	11,13
Yandex Bot Disallow	442	5,95
Microsoft BingBot Disallow	422	5,68
Alexa Bot Disallow	413	5,56
Soso Bot Disallow	413	5,56
Yahoo Slurp Disallow	412	5,55
Robots Disallow	271	3,65
Google Media Partners Disallow	203 ^*	2,73
Google AdsBot Disallow	82	1,1
Ahrefs Bot Disallow	74	1
GPTBot Disallow	60	0,81
Google AdsBot Mobile Disallow	56	0,75
Semrush Bot Disallow	53	0,71
Majestic Bot Disallow	51	0,69
Common Crawl Bot Disallow	40	0,54
Anthropic Claude Bot Disallow	35	0,47
Google Bard Disallow	35	0,47
AppleBot Disallow	32	0,43
Facebook Bot Disallow	32	0,43
Cohere AI Disallow	29	0,39
Apple AI Bot Disallow	28	0,38
Perplexity	27	0,36
Google AdsBot Mobile Apps Disallow	27	0,36
360 Bot Disallow	14	0,19
ByteDance Bytespider Disallow	9	0,12
Exalead Bot Disallow	7	0,09
Webz Omgili Disallow	7	0,09
Microsoft MSNBot Disallow	6	0,08
Google Image Bot Disallow	5	0,07
BuiltWithBot Disallow	4	0,05
Diffbot Disallow	3	0,04
DuckDuckGo Bot Disallow	3	0,04
Google News Bot Disallow	2	0,03
Google Video Bot Disallow	2	0,03
Google Favicon Disallow	2	0,03
Google API Bot Disallow	2	0,03
Google DuplexWeb Disallow	2	0,03
YOU Disallow	1	0,01
Youdao Bot Disallow	1	0,01
Pipl Disallow	1	0,01