Google’s John Mueller answers a question about using robots.txt to block special files, including .css and .htacess. This topic was discussed in some detail in the latest edition of the Ask Google Webmasters video series on YouTube. Here is the question that was submitted: “Regarding robots.txt, should I ‘disallow: /*.css$’, ‘disallow: /php.ini’, or even ‘disallow: /.htaccess’?” In response, Mueller says Google can’t stop site owners from disallowing those files. Although… Source link
Read More »Blocking Special Files in Robots.txt
