Robots.txt Usage Distribution in Bulgaria

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Ahrefs Bot Disallow	4,204	11.49
Googlebot Disallow	3,680	10.06
Google AdsBot Disallow	3,203	8.75
Robots Disallow	2,739	7.49
Majestic Bot Disallow	2,362	6.46
Google AdsBot Mobile Disallow	1,979	5.41
Yandex Bot Disallow	1,884	5.15
Baidu Bot Disallow	1,776	4.85
Microsoft BingBot Disallow	1,598	4.37
Semrush Bot Disallow	1,408	3.85
Sogou Bot Disallow	1,371	3.75
Alexa Bot Disallow	1,359	3.71
Yahoo Slurp Disallow	1,196	3.27
Soso Bot Disallow	1,164	3.18
Common Crawl Bot Disallow	716	1.96
Some Robots Disallow	589	1.61
GPTBot Disallow	582	1.59
Anthropic Claude Bot Disallow	455	1.24
Google AdsBot Mobile Apps Disallow	367	1
Google Bard Disallow	362	0.99
Exalead Bot Disallow	359	0.98
AppleBot Disallow	354	0.97
Facebook Bot Disallow	349	0.95
Cohere AI Disallow	308	0.84
Apple AI Bot Disallow	302	0.83
Perplexity	299	0.82
Microsoft MSNBot Disallow	260	0.71
Google Media Partners Disallow	242 ^*	0.66
Google Image Bot Disallow	175	0.48
360 Bot Disallow	160	0.44
ByteDance Bytespider Disallow	138	0.38
BuiltWithBot Disallow	98	0.27
YOU Disallow	77	0.21
Google News Bot Disallow	71	0.19
Google Video Bot Disallow	70	0.19
DuckDuckGo Bot Disallow	62	0.17
Webz Omgili Disallow	57	0.16
Google Favicon Disallow	56	0.15
Google API Bot Disallow	53	0.14
Google DuplexWeb Disallow	53	0.14
Diffbot Disallow	23	0.06
Youdao Bot Disallow	12	0.03
Pipl Disallow	12	0.03
Google FeedFetcher Disallow	1	0