Robots.txt Usage Distribution in Libya

Top In Robots.txt Usage Distribution in Libya

Technology	Websites	%
Google Media Partners Disallow	80 *	37.38
Robots Disallow	38	17.76
Ahrefs Bot Disallow	8	3.74
GPTBot Disallow	7	3.27
Common Crawl Bot Disallow	7	3.27
Anthropic Claude Bot Disallow	7	3.27
Google Bard Disallow	7	3.27
Facebook Bot Disallow	7	3.27
Perplexity	7	3.27
AppleBot Disallow	6	2.8
Cohere AI Disallow	6	2.8
Apple AI Bot Disallow	6	2.8
Google AdsBot Mobile Apps Disallow	6	2.8
Yandex Bot Disallow	5	2.34
Microsoft BingBot Disallow	3	1.4
Alexa Bot Disallow	3	1.4
Yahoo Slurp Disallow	3	1.4
Soso Bot Disallow	3	1.4
Semrush Bot Disallow	2	0.93
Majestic Bot Disallow	1	0.47
ByteDance Bytespider Disallow	1	0.47
Google Image Bot Disallow	1	0.47

214 Detections

of Robots.txt in Libya. Last updated 21st Apr 2025.

Google Media Partners Disallow is currently the most popular technology in this category.