Usage Distribution in the Top 1 Million Sites

Top In Usage Distribution in the Top 1 Million Sites

Technology	Websites	%
GPTBot Disallow	26,583	2.66
Alexa Bot Disallow	22,636	2.26
Common Crawl Bot Disallow	18,917	1.89
Anthropic Claude Bot Disallow	12,018	1.2
Google Bard Disallow	11,523	1.15
AmazonBot Disallow	10,501	1.05
Facebook Bot Disallow	8,165	0.82
ByteDance Bytespider Disallow	6,483	0.65
Perplexity	5,289	0.53
Cohere AI Disallow	5,231	0.52
Webz Omgili Disallow	5,131	0.51
Apple AI Bot Disallow	4,910	0.49
Diffbot Disallow	2,126	0.21
YOU Disallow	1,449	0.14

140,962 Detections

of in the Top 1 Million Sites. Last updated 11th Sep 2024.

GPTBot Disallow is currently the most popular technology in this category.