Robots.txt Distribuzione dell'utilizzo In Tacchino

Distribuzione per i siti Web che utilizzano Robots.txt tecnologie

Tecnologia	Siti web	%
Googlebot Disallow	53.250	23,04
Google AdsBot Disallow	49.536	21,43
Google AdsBot Mobile Disallow	41.973	18,16
Ahrefs Bot Disallow	11.583	5,01
Robots Disallow	8.225	3,56
Alexa Bot Disallow	7.107	3,07
Baidu Bot Disallow	6.729	2,91
GPTBot Disallow	5.765	2,49
Majestic Bot Disallow	5.290	2,29
Semrush Bot Disallow	4.350	1,88
Sogou Bot Disallow	3.895	1,69
Soso Bot Disallow	3.885	1,68
Yandex Bot Disallow	3.672	1,59
Microsoft BingBot Disallow	2.875	1,24
Yahoo Slurp Disallow	2.797	1,21
Some Robots Disallow	2.622	1,13
Google Image Bot Disallow	2.043	0,88
Exalead Bot Disallow	1.971	0,85
360 Bot Disallow	1.540	0,67
Common Crawl Bot Disallow	1.481	0,64
Google Media Partners Disallow	1.260 ^*	0,55
Anthropic Claude Bot Disallow	1.186	0,51
Google Bard Disallow	989	0,43
AppleBot Disallow	891	0,39
Facebook Bot Disallow	861	0,37
Google AdsBot Mobile Apps Disallow	828	0,36
Apple AI Bot Disallow	801	0,35
Cohere AI Disallow	788	0,34
Perplexity	744	0,32
ByteDance Bytespider Disallow	471	0,2
Microsoft MSNBot Disallow	445	0,19
Webz Omgili Disallow	275	0,12
BuiltWithBot Disallow	252	0,11
Google Video Bot Disallow	99	0,04
DuckDuckGo Bot Disallow	99	0,04
Youdao Bot Disallow	97	0,04
Google News Bot Disallow	94	0,04
Diffbot Disallow	81	0,04
Google Favicon Disallow	73	0,03
Google API Bot Disallow	72	0,03
Google DuplexWeb Disallow	70	0,03
YOU Disallow	43	0,02
Pipl Disallow	15	0,01
Google FeedFetcher Disallow	10	0
Google Read Aloud Disallow	1	0
Google Web Light Disallow	1	0