of Robots.txt in the Top 100k Sites. Last updated 14th Nov 2024.
Google Media Partners Disallow is currently the most popular technology in this category.