Robots.txt Distribuzione dell'utilizzo In Brasile

Distribuzione per i siti Web che utilizzano Robots.txt tecnologie

Tecnologia	Siti web	%
Google AdsBot Disallow	219.099	17,84
Googlebot Disallow	194.855	15,86
Google AdsBot Mobile Disallow	166.149	13,53
Robots Disallow	111.264	9,06
Ahrefs Bot Disallow	92.550	7,53
Baidu Bot Disallow	54.969	4,48
Majestic Bot Disallow	54.342	4,42
Yandex Bot Disallow	47.345	3,85
Yahoo Slurp Disallow	47.041	3,83
Exalead Bot Disallow	43.159	3,51
GPTBot Disallow	33.931	2,76
Anthropic Claude Bot Disallow	26.593	2,16
Some Robots Disallow	22.708	1,85
Semrush Bot Disallow	12.380	1,01
Microsoft BingBot Disallow	11.243	0,92
Alexa Bot Disallow	10.575	0,86
Sogou Bot Disallow	9.470	0,77
Google Image Bot Disallow	9.014	0,73
Soso Bot Disallow	8.114	0,66
AppleBot Disallow	7.503	0,61
Common Crawl Bot Disallow	6.386	0,52
Google Media Partners Disallow	5.024 ^*	0,41
Google Bard Disallow	4.211	0,34
Facebook Bot Disallow	3.913	0,32
Apple AI Bot Disallow	3.650	0,3
Google AdsBot Mobile Apps Disallow	3.555	0,29
Perplexity	3.411	0,28
Cohere AI Disallow	3.333	0,27
Microsoft MSNBot Disallow	2.598	0,21
360 Bot Disallow	1.831	0,15
ByteDance Bytespider Disallow	1.447	0,12
DuckDuckGo Bot Disallow	1.406	0,11
BuiltWithBot Disallow	1.013	0,08
Webz Omgili Disallow	799	0,07
Google FeedFetcher Disallow	640	0,05
Youdao Bot Disallow	632	0,05
YOU Disallow	593	0,05
Google News Bot Disallow	346	0,03
Google API Bot Disallow	317	0,03
Google Video Bot Disallow	287	0,02
Google Favicon Disallow	223	0,02
Google DuplexWeb Disallow	173	0,01
Diffbot Disallow	133	0,01
Pipl Disallow	93	0,01
Google Read Aloud Disallow	10	0
Google Web Light Disallow	3	0