Robots.txt Répartition des utilisations en Gambie

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Google AdsBot Disallow	72	13,53
Google AdsBot Mobile Disallow	68	12,78
Googlebot Disallow	67	12,59
Robots Disallow	51	9,59
GPTBot Disallow	20	13,99
Baidu Bot Disallow	16	3,01
AppleBot Disallow	16	3,01
Common Crawl Bot Disallow	15	10,49
Anthropic Claude Bot Disallow	15	10,49
Google Bard Disallow	15	10,49
Facebook Bot Disallow	15	10,49
Cohere AI Disallow	15	10,49
Apple AI Bot Disallow	15	10,49
Perplexity	15	10,49
Google AdsBot Mobile Apps Disallow	15	2,82
Google Media Partners Disallow	13 ^*	2,44
Yandex Bot Disallow	12	2,26
Ahrefs Bot Disallow	12	2,26
Microsoft BingBot Disallow	8	1,5
Yahoo Slurp Disallow	8	1,5
Soso Bot Disallow	7	1,32
Alexa Bot Disallow	6	4,2
Semrush Bot Disallow	6	1,13
Majestic Bot Disallow	5	0,94
Google Image Bot Disallow	3	0,56
ByteDance Bytespider Disallow	2	1,4
Microsoft MSNBot Disallow	2	0,38
Webz Omgili Disallow	2	1,4
Diffbot Disallow	2	1,4
YOU Disallow	2	1,4
Google News Bot Disallow	2	0,38
Google Video Bot Disallow	2	0,38
DuckDuckGo Bot Disallow	2	0,38
Google Favicon Disallow	2	0,38
Google API Bot Disallow	2	0,38
Google DuplexWeb Disallow	2	0,38