Robots.txt Répartition de l'utilisation dans Oman

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Google AdsBot Disallow	453	17,82
Googlebot Disallow	416	16,37
Robots Disallow	206	8,1
Ahrefs Bot Disallow	129	5,07
Baidu Bot Disallow	97	3,82
Yandex Bot Disallow	79	3,11
Common Crawl Bot Disallow	78	3,07
GPTBot Disallow	75	2,95
Yahoo Slurp Disallow	66	2,6
Sogou Bot Disallow	66	2,6
Google Bard Disallow	64	2,52
Google AdsBot Mobile Apps Disallow	64	2,52
Soso Bot Disallow	63	2,48
Alexa Bot Disallow	62	2,44
Microsoft BingBot Disallow	60	2,36
AppleBot Disallow	60	2,36
Anthropic Claude Bot Disallow	60	2,36
Facebook Bot Disallow	59	2,32
Some Robots Disallow	58	2,28
Cohere AI Disallow	58	2,28
Apple AI Bot Disallow	57	2,24
Perplexity	57	2,24
Majestic Bot Disallow	47	1,85
Semrush Bot Disallow	34	1,34
Microsoft MSNBot Disallow	14	0,55
Exalead Bot Disallow	12	0,47
360 Bot Disallow	11	0,43
BuiltWithBot Disallow	9	0,35
Google Image Bot Disallow	8	0,31
Google News Bot Disallow	5	0,2
ByteDance Bytespider Disallow	4	0,16
Google Video Bot Disallow	4	0,16
DuckDuckGo Bot Disallow	3	0,12
Youdao Bot Disallow	2	0,08
Webz Omgili Disallow	1	0,04
Google Favicon Disallow	1	0,04