Robots.txt Répartition des utilisations en Namibie

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Google AdsBot Disallow	535	21,64
Googlebot Disallow	498	20,15
Google AdsBot Mobile Disallow	490	19,82
Robots Disallow	135	5,46
Ahrefs Bot Disallow	49	1,98
Common Crawl Bot Disallow	48	1,94
GPTBot Disallow	44	1,78
Google Bard Disallow	44	1,78
Google Media Partners Disallow	44 ^*	1,78
Some Robots Disallow	43	1,74
Yandex Bot Disallow	43	1,74
AppleBot Disallow	43	1,74
Anthropic Claude Bot Disallow	42	1,7
Facebook Bot Disallow	42	1,7
Apple AI Bot Disallow	42	1,7
Cohere AI Disallow	40	1,62
Perplexity	39	1,58
Google AdsBot Mobile Apps Disallow	38	1,54
Microsoft BingBot Disallow	37	1,5
Alexa Bot Disallow	37	1,5
Yahoo Slurp Disallow	34	1,38
Soso Bot Disallow	29	1,17
Majestic Bot Disallow	21	0,85
Semrush Bot Disallow	13	0,53
Microsoft MSNBot Disallow	9	0,36
Google Image Bot Disallow	8	0,32
Exalead Bot Disallow	6	0,24
ByteDance Bytespider Disallow	4	0,16
Webz Omgili Disallow	4	0,16
Diffbot Disallow	3	0,12
360 Bot Disallow	2	0,08
BuiltWithBot Disallow	1	0,04
YOU Disallow	1	0,04
Google News Bot Disallow	1	0,04
Google Video Bot Disallow	1	0,04
DuckDuckGo Bot Disallow	1	0,04
Google Favicon Disallow	1	0,04