Robots.txt Distribuzione dell'utilizzo In Tacchino

Distribuzione per i siti Web che utilizzano Robots.txt tecnologie

Tecnologia	Siti web	%
Googlebot Disallow	52.865	22,82
Google AdsBot Disallow	48.575	20,97
Google AdsBot Mobile Disallow	40.857	17,64
Ahrefs Bot Disallow	11.597	5,01
Alexa Bot Disallow	7.285	3,15
Baidu Bot Disallow	7.124	3,08
Robots Disallow	7.073	3,05
GPTBot Disallow	5.678	2,45
Majestic Bot Disallow	5.065	2,19
Sogou Bot Disallow	4.864	2,1
Semrush Bot Disallow	4.313	1,86
Soso Bot Disallow	4.058	1,75
Yandex Bot Disallow	3.832	1,65
Microsoft BingBot Disallow	3.072	1,33
Some Robots Disallow	3.049	1,32
Yahoo Slurp Disallow	2.920	1,26
Google Media Partners Disallow	2.853 ^*	1,23
Google Image Bot Disallow	1.897	0,82
Exalead Bot Disallow	1.873	0,81
360 Bot Disallow	1.498	0,65
Common Crawl Bot Disallow	1.480	0,64
Anthropic Claude Bot Disallow	1.308	0,56
Google Bard Disallow	1.053	0,45
AppleBot Disallow	981	0,42
Facebook Bot Disallow	923	0,4
Apple AI Bot Disallow	863	0,37
Cohere AI Disallow	852	0,37
Perplexity	819	0,35
Google AdsBot Mobile Apps Disallow	806	0,35
ByteDance Bytespider Disallow	568	0,25
Microsoft MSNBot Disallow	369	0,16
Webz Omgili Disallow	300	0,13
BuiltWithBot Disallow	250	0,11
Diffbot Disallow	107	0,05
DuckDuckGo Bot Disallow	91	0,04
Google Video Bot Disallow	82	0,04
Google News Bot Disallow	77	0,03
Youdao Bot Disallow	74	0,03
Google API Bot Disallow	60	0,03
Google Favicon Disallow	59	0,03
YOU Disallow	58	0,03
Google DuplexWeb Disallow	58	0,03
Pipl Disallow	16	0,01
Google FeedFetcher Disallow	12	0,01
Google Read Aloud Disallow	2	0
Google Web Light Disallow	1	0