Robots.txt Nutzungsverteilung In Fidschi

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Google AdsBot Disallow	206	15,07
Google AdsBot Mobile Disallow	174	12,73
Googlebot Disallow	130	9,51
GPTBot Disallow	64	4,68
Common Crawl Bot Disallow	64	4,68
AppleBot Disallow	64	4,68
Anthropic Claude Bot Disallow	63	4,61
Google Bard Disallow	63	4,61
Facebook Bot Disallow	63	4,61
Apple AI Bot Disallow	63	4,61
Perplexity	63	4,61
Google AdsBot Mobile Apps Disallow	63	4,61
Cohere AI Disallow	62	4,54
Robots Disallow	51	3,73
Ahrefs Bot Disallow	35	2,56
Google Media Partners Disallow	34 ^*	2,49
Baidu Bot Disallow	16	1,17
Some Robots Disallow	15	1,1
Yandex Bot Disallow	14	1,02
Alexa Bot Disallow	11	0,8
Microsoft BingBot Disallow	9	0,66
Yahoo Slurp Disallow	9	0,66
Soso Bot Disallow	8	0,59
Majestic Bot Disallow	5	0,37
Semrush Bot Disallow	3	0,22
ByteDance Bytespider Disallow	2	0,15
Microsoft MSNBot Disallow	2	0,15
Google Image Bot Disallow	2	0,15
Exalead Bot Disallow	1	0,07
Webz Omgili Disallow	1	0,07
Google News Bot Disallow	1	0,07
Google Video Bot Disallow	1	0,07
YOU Disallow	1	0,07
DuckDuckGo Bot Disallow	1	0,07
Google Favicon Disallow	1	0,07
Google API Bot Disallow	1	0,07
Google DuplexWeb Disallow	1	0,07