Robots.txt Nutzungsverteilung In Trinidad und Tobago

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Google AdsBot Disallow	243	10,04
Google AdsBot Mobile Disallow	215	8,88
Googlebot Disallow	147	6,07
GPTBot Disallow	134	5,53
AppleBot Disallow	133	5,49
Common Crawl Bot Disallow	132	5,45
Google Bard Disallow	132	5,45
Anthropic Claude Bot Disallow	131	5,41
Facebook Bot Disallow	131	5,41
Apple AI Bot Disallow	131	5,41
Cohere AI Disallow	129	5,33
Perplexity	129	5,33
Google AdsBot Mobile Apps Disallow	129	5,33
Robots Disallow	118	4,87
Ahrefs Bot Disallow	42	1,73
Yahoo Slurp Disallow	41	1,69
Yandex Bot Disallow	40	1,65
Baidu Bot Disallow	38	1,57
Microsoft BingBot Disallow	34	1,4
Alexa Bot Disallow	32	1,32
Soso Bot Disallow	25	1,03
Sogou Bot Disallow	23	0,95
Some Robots Disallow	22	0,91
Microsoft MSNBot Disallow	15	0,62
Majestic Bot Disallow	14	0,58
Semrush Bot Disallow	14	0,58
Google Image Bot Disallow	13	0,54
Exalead Bot Disallow	5	0,21
Webz Omgili Disallow	5	0,21
Google News Bot Disallow	4	0,17
Google Video Bot Disallow	4	0,17
Google Favicon Disallow	4	0,17
ByteDance Bytespider Disallow	3	0,12
DuckDuckGo Bot Disallow	3	0,12
360 Bot Disallow	2	0,08
Diffbot Disallow	1	0,04
YOU Disallow	1	0,04
Google API Bot Disallow	1	0,04
Google DuplexWeb Disallow	1	0,04