Robots.txt Distribuzione dell'utilizzo In Isole Salomone

Distribuzione per i siti Web che utilizzano Robots.txt tecnologie

Tecnologia	Siti web	%
Robots Disallow	151	39,84
Googlebot Disallow	18	4,75
Google Media Partners Disallow	17 ^*	4,49
GPTBot Disallow	14	3,69
Common Crawl Bot Disallow	13	3,43
Anthropic Claude Bot Disallow	12	3,17
Facebook Bot Disallow	12	3,17
AppleBot Disallow	11	2,9
Google Bard Disallow	11	2,9
Apple AI Bot Disallow	11	2,9
Cohere AI Disallow	11	2,9
Perplexity	10	2,64
Baidu Bot Disallow	9	2,37
Sogou Bot Disallow	9	2,37
Google AdsBot Mobile Apps Disallow	9	2,37
Soso Bot Disallow	8	2,11
Yandex Bot Disallow	7	1,85
Alexa Bot Disallow	7	1,85
Yahoo Slurp Disallow	7	1,85
Microsoft BingBot Disallow	6	1,58
Ahrefs Bot Disallow	6	1,58
Semrush Bot Disallow	3	0,79
ByteDance Bytespider Disallow	3	0,79
Webz Omgili Disallow	3	0,79
Majestic Bot Disallow	2	0,53
Exalead Bot Disallow	2	0,53
360 Bot Disallow	2	0,53
BuiltWithBot Disallow	2	0,53
Diffbot Disallow	2	0,53
Microsoft MSNBot Disallow	1	0,26