Robot.txt Distribuzione dell'utilizzo nel Canada

Distribuzione per siti web che utilizzano Robot.txt tecnologie

Tecnologia	Siti web	%
Google AdsBot Disallow	215.305	13,06
Google AdsBot Mobile Disallow	161.571	9,8
Googlebot Disallow	144.875	8,79
GPTBot Disallow	73.612	4,47
Common Crawl Bot Disallow	71.380	4,33
Anthropic Claude Bot Disallow	69.813	4,24
Facebook Bot Disallow	68.955	4,18
AppleBot Disallow	68.908	4,18
Google Bard Disallow	68.894	4,18
Cohere AI Disallow	67.517	4,1
Perplexity	67.513	4,1
Apple AI Bot Disallow	67.435	4,09
Google AdsBot Mobile Apps Disallow	65.472	3,97
Ahrefs Bot Disallow	65.393	3,97
Baidu Bot Disallow	48.674	2,95
Sogou Bot Disallow	42.802	2,6
Robots Disallow	41.347	2,51
Yandex Bot Disallow	35.144	2,13
Microsoft BingBot Disallow	28.781	1,75
Alexa Bot Disallow	27.977	1,7
Yahoo Slurp Disallow	27.732	1,68
Some Robots Disallow	27.712	1,68
Soso Bot Disallow	25.184	1,53
Semrush Bot Disallow	14.635	0,89
Majestic Bot Disallow	14.307	0,87
ByteDance Bytespider Disallow	5.885	0,36
Microsoft MSNBot Disallow	4.634	0,28
Google Media Partners Disallow	4.498 ^*	0,27
Google Image Bot Disallow	3.871	0,23
Webz Omgili Disallow	3.500	0,21
Exalead Bot Disallow	3.009	0,18
Diffbot Disallow	1.839	0,11
360 Bot Disallow	1.562	0,09
Google News Bot Disallow	1.211	0,07
Google Video Bot Disallow	1.084	0,07
Google Favicon Disallow	1.045	0,06
YOU Disallow	907	0,06
BuiltWithBot Disallow	827	0,05
DuckDuckGo Bot Disallow	735	0,04
Google API Bot Disallow	619	0,04
Google DuplexWeb Disallow	605	0,04
Google FeedFetcher Disallow	386	0,02
Google Read Aloud Disallow	382	0,02
Pipl Disallow	379	0,02
Google Web Light Disallow	373	0,02
Youdao Bot Disallow	176	0,01
LLMS Text	1	0