Robot.txt Distribuzione dell'utilizzo in Lussemburgo

Distribuzione per siti web che utilizzano Robot.txt tecnologie

Tecnologia	Siti web	%
Googlebot Disallow	6.693	13,54
Baidu Bot Disallow	3.739	7,56
Sogou Bot Disallow	3.502	7,09
Google AdsBot Disallow	3.461	7
Yandex Bot Disallow	3.316	6,71
Microsoft BingBot Disallow	3.216	6,51
Yahoo Slurp Disallow	3.198	6,47
Alexa Bot Disallow	3.189	6,45
Soso Bot Disallow	3.105	6,28
Google AdsBot Mobile Disallow	3.051	6,17
Robots Disallow	2.312	4,68
Majestic Bot Disallow	1.198	2,42
Anthropic Claude Bot Disallow	1.048	2,12
Ahrefs Bot Disallow	994	2,01
GPTBot Disallow	771	1,56
Semrush Bot Disallow	599	1,21
Common Crawl Bot Disallow	557	1,13
Google Bard Disallow	528	1,07
Google AdsBot Mobile Apps Disallow	514	1,04
Some Robots Disallow	509	1,03
AppleBot Disallow	509	1,03
Facebook Bot Disallow	493	1
Cohere AI Disallow	477	0,97
Perplexity	475	0,96
Apple AI Bot Disallow	470	0,95
Google Media Partners Disallow	332 ^*	0,67
Microsoft MSNBot Disallow	152	0,31
Google Image Bot Disallow	138	0,28
Google News Bot Disallow	91	0,18
DuckDuckGo Bot Disallow	89	0,18
ByteDance Bytespider Disallow	87	0,18
Google Video Bot Disallow	87	0,18
Google Favicon Disallow	84	0,17
Webz Omgili Disallow	80	0,16
Google API Bot Disallow	79	0,16
Google DuplexWeb Disallow	79	0,16
Exalead Bot Disallow	70	0,14
Diffbot Disallow	45	0,09
360 Bot Disallow	34	0,07
YOU Disallow	25	0,05
BuiltWithBot Disallow	11	0,02
Pipl Disallow	10	0,02
Youdao Bot Disallow	8	0,02
Google Read Aloud Disallow	1	0
LLMS Text	1	0
Google Web Light Disallow	1	0