Robot.txt Distribuzione dell'utilizzo in Norvegia

Distribuzione per siti web che utilizzano Robot.txt tecnologie

Tecnologia	Siti web	%
Google AdsBot Disallow	33.053	10,92
Googlebot Disallow	28.231	9,33
Google AdsBot Mobile Disallow	27.033	8,93
Robots Disallow	20.068	6,63
Baidu Bot Disallow	11.538	3,81
GPTBot Disallow	10.661	3,52
Sogou Bot Disallow	10.576	3,5
Yandex Bot Disallow	10.525	3,48
Anthropic Claude Bot Disallow	10.244	3,39
Common Crawl Bot Disallow	10.177	3,36
Microsoft BingBot Disallow	10.092	3,34
Yahoo Slurp Disallow	10.053	3,32
Google Bard Disallow	10.010	3,31
Alexa Bot Disallow	9.992	3,3
AppleBot Disallow	9.769	3,23
Facebook Bot Disallow	9.768	3,23
Perplexity	9.714	3,21
Soso Bot Disallow	9.682	3,2
Apple AI Bot Disallow	9.630	3,18
Cohere AI Disallow	9.595	3,17
Google AdsBot Mobile Apps Disallow	8.996	2,97
Ahrefs Bot Disallow	7.181	2,37
Majestic Bot Disallow	2.869	0,95
Some Robots Disallow	2.570	0,85
Semrush Bot Disallow	2.475	0,82
Google Media Partners Disallow	1.376 ^*	0,45
ByteDance Bytespider Disallow	1.126	0,37
Webz Omgili Disallow	939	0,31
Diffbot Disallow	770	0,25
Microsoft MSNBot Disallow	662	0,22
YOU Disallow	655	0,22
Google Image Bot Disallow	561	0,19
Exalead Bot Disallow	442	0,15
Google Video Bot Disallow	400	0,13
Google News Bot Disallow	264	0,09
Google Favicon Disallow	192	0,06
DuckDuckGo Bot Disallow	159	0,05
Google API Bot Disallow	145	0,05
Google DuplexWeb Disallow	145	0,05
360 Bot Disallow	102	0,03
BuiltWithBot Disallow	73	0,02
Google FeedFetcher Disallow	44	0,01
Pipl Disallow	21	0,01
Youdao Bot Disallow	4	0
LLMS Text	1	0