Robots.txt Répartition de l'utilisation dans Madagascar

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Robots Disallow	189	17,91
Googlebot Disallow	136	12,89
Google AdsBot Disallow	112	10,62
Google AdsBot Mobile Disallow	101	9,57
Baidu Bot Disallow	34	3,22
Common Crawl Bot Disallow	29	13,62
GPTBot Disallow	28	13,15
Microsoft BingBot Disallow	27	2,56
Ahrefs Bot Disallow	26	2,46
Yandex Bot Disallow	25	2,37
Yahoo Slurp Disallow	25	2,37
Google Bard Disallow	23	10,8
Google AdsBot Mobile Apps Disallow	23	2,18
Alexa Bot Disallow	21	9,86
Anthropic Claude Bot Disallow	21	9,86
Facebook Bot Disallow	21	9,86
Cohere AI Disallow	21	9,86
AppleBot Disallow	20	1,9
Apple AI Bot Disallow	20	9,39
Perplexity	20	9,39
Sogou Bot Disallow	18	1,71
Soso Bot Disallow	18	1,71
Majestic Bot Disallow	18	1,71
Google Media Partners Disallow	16 ^*	1,52
Semrush Bot Disallow	16	1,52
DuckDuckGo Bot Disallow	7	0,66
Google Image Bot Disallow	6	0,57
Exalead Bot Disallow	5	0,47
Microsoft MSNBot Disallow	5	0,47
ByteDance Bytespider Disallow	3	1,41
360 Bot Disallow	3	0,28
BuiltWithBot Disallow	3	0,28
Google News Bot Disallow	3	0,28
Google Video Bot Disallow	2	0,19
Google Favicon Disallow	2	0,19
Google API Bot Disallow	2	0,19
Google DuplexWeb Disallow	2	0,19
YOU Disallow	2	0,94
Webz Omgili Disallow	1	0,47
Diffbot Disallow	1	0,47