Robots.txt Répartition des utilisations en Israël

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Google AdsBot Disallow	20 830	25,61
Googlebot Disallow	18 896	23,23
Google AdsBot Mobile Disallow	18 274	22,47
Ahrefs Bot Disallow	3 409	4,19
Robots Disallow	2 995	3,68
Majestic Bot Disallow	1 840	2,26
Semrush Bot Disallow	1 325	1,63
Yandex Bot Disallow	1 146	1,41
Baidu Bot Disallow	871	1,07
Some Robots Disallow	737	0,91
GPTBot Disallow	732	0,9
Common Crawl Bot Disallow	726	0,89
Microsoft BingBot Disallow	687	0,84
AppleBot Disallow	677	0,83
Anthropic Claude Bot Disallow	661	0,81
Sogou Bot Disallow	645	0,79
Google Bard Disallow	643	0,79
Apple AI Bot Disallow	631	0,78
Facebook Bot Disallow	624	0,77
Alexa Bot Disallow	602	0,74
Perplexity	601	0,74
Cohere AI Disallow	587	0,72
Soso Bot Disallow	581	0,71
Google AdsBot Mobile Apps Disallow	576	0,71
Yahoo Slurp Disallow	536	0,66
Exalead Bot Disallow	329	0,4
Google Media Partners Disallow	249 ^*	0,31
Google Image Bot Disallow	172	0,21
Microsoft MSNBot Disallow	162	0,2
Webz Omgili Disallow	137	0,17
YOU Disallow	106	0,13
360 Bot Disallow	68	0,08
ByteDance Bytespider Disallow	67	0,08
BuiltWithBot Disallow	40	0,05
DuckDuckGo Bot Disallow	25	0,03
Google News Bot Disallow	23	0,03
Diffbot Disallow	22	0,03
LLMS Text	21	0,03
Google Video Bot Disallow	20	0,02
Google Favicon Disallow	20	0,02
Google API Bot Disallow	13	0,02
Google DuplexWeb Disallow	13	0,02
Pipl Disallow	6	0,01
Youdao Bot Disallow	5	0,01
Google Read Aloud Disallow	1	0