Robots.txt Distribuzione dell'utilizzo In Cipro

Distribuzione per i siti Web che utilizzano Robots.txt tecnologie

Tecnologia	Siti web	%
Google AdsBot Disallow	1.084	13,52
Googlebot Disallow	847	10,57
Google AdsBot Mobile Disallow	843	10,52
Robots Disallow	470	5,86
Ahrefs Bot Disallow	359	4,48
GPTBot Disallow	323	13,11
Common Crawl Bot Disallow	312	12,66
Google AdsBot Mobile Apps Disallow	287	3,58
Anthropic Claude Bot Disallow	277	11,24
Google Bard Disallow	276	11,2
Facebook Bot Disallow	273	11,08
AppleBot Disallow	271	3,38
Apple AI Bot Disallow	266	10,8
Cohere AI Disallow	266	10,8
Perplexity	263	10,67
Yandex Bot Disallow	194	2,42
Baidu Bot Disallow	182	2,27
Alexa Bot Disallow	144	5,84
Some Robots Disallow	141	1,76
Yahoo Slurp Disallow	138	1,72
Microsoft BingBot Disallow	127	1,58
Sogou Bot Disallow	117	1,46
Soso Bot Disallow	112	1,4
Semrush Bot Disallow	92	1,15
Majestic Bot Disallow	88	1,1
Google Media Partners Disallow	81 ^*	1,01
Microsoft MSNBot Disallow	50	0,62
Exalead Bot Disallow	21	0,26
Google Image Bot Disallow	20	0,25
ByteDance Bytespider Disallow	16	0,65
360 Bot Disallow	15	0,19
Webz Omgili Disallow	12	0,49
Google News Bot Disallow	8	0,1
BuiltWithBot Disallow	7	0,09
Google Video Bot Disallow	7	0,09
DuckDuckGo Bot Disallow	5	0,06
YOU Disallow	5	0,2
Google Favicon Disallow	4	0,05
Google API Bot Disallow	4	0,05
Google DuplexWeb Disallow	4	0,05
Youdao Bot Disallow	2	0,02
Diffbot Disallow	1	0,04
Pipl Disallow	1	0,01