Robots.txt Nutzungsverteilung In Kanada

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Google AdsBot Disallow	307.060	14
Google AdsBot Mobile Disallow	232.321	10,6
Googlebot Disallow	194.199	8,86
GPTBot Disallow	107.221	13
Common Crawl Bot Disallow	103.201	12,52
Google Bard Disallow	96.090	11,65
Google AdsBot Mobile Apps Disallow	95.984	4,38
Anthropic Claude Bot Disallow	95.796	11,62
Facebook Bot Disallow	94.475	11,46
AppleBot Disallow	93.819	4,28
Apple AI Bot Disallow	92.357	11,2
Ahrefs Bot Disallow	91.394	4,17
Cohere AI Disallow	90.778	11,01
Perplexity	90.473	10,97
Baidu Bot Disallow	57.154	2,61
Robots Disallow	42.611	1,94
Yandex Bot Disallow	40.333	1,84
Microsoft BingBot Disallow	31.683	1,44
Yahoo Slurp Disallow	30.971	1,41
Alexa Bot Disallow	30.910	3,75
Some Robots Disallow	30.909	1,41
Sogou Bot Disallow	29.571	1,35
Soso Bot Disallow	27.160	1,24
Majestic Bot Disallow	20.927	0,95
Semrush Bot Disallow	20.435	0,93
ByteDance Bytespider Disallow	7.477	0,91
Microsoft MSNBot Disallow	6.958	0,32
Webz Omgili Disallow	5.701	0,69
Google Image Bot Disallow	5.513	0,25
Exalead Bot Disallow	4.266	0,19
Google Media Partners Disallow	3.134 ^*	0,14
360 Bot Disallow	2.557	0,12
Google News Bot Disallow	1.575	0,07
Google Video Bot Disallow	1.445	0,07
Diffbot Disallow	1.322	0,16
BuiltWithBot Disallow	1.101	0,05
Google Favicon Disallow	990	0,05
DuckDuckGo Bot Disallow	746	0,03
YOU Disallow	473	0,06
Pipl Disallow	442	0,02
Youdao Bot Disallow	419	0,02
Google API Bot Disallow	321	0,01
Google DuplexWeb Disallow	307	0,01
Google FeedFetcher Disallow	18	0
Google Read Aloud Disallow	3	0