[wp-trac] [WordPress Trac] #18465: Prevent search engines from indexing wp-admin and wp-includes (was: robots.txt should tell Google to not index wp-admin and wp-includes)

WordPress Trac wp-trac at lists.automattic.com
Wed Dec 21 12:07:07 UTC 2011

#18465: Prevent search engines from indexing wp-admin and wp-includes
 Reporter:  Viper007Bond  |       Owner:  ryan
     Type:  enhancement   |      Status:  reopened
 Priority:  lowest        |   Milestone:  3.3
Component:  General       |     Version:  3.2.1
 Severity:  trivial       |  Resolution:
 Keywords:  has-patch     |
Changes (by joostdevalk):

 * status:  closed => reopened
 * resolution:  fixed =>


 This is a valid problem but the "fix" doesn't actually fix it. While the
 addition to robots.txt blocks the crawler from opening the URL, a URL that
 cannot be opened CAN still be listed in the index if Google finds enough
 links pointing to it, see the note on
 this Google help page]. Example of this can be seen on my Dutch domain:


 The solution is to not exclude the admin directory in robots.txt, but to
 send an X-Robots-Tag HTTP header of value noindex (the HTTP version of a
 robots meta tag) for the files in admin and for admin-ajax.php, will add a

Ticket URL: <http://core.trac.wordpress.org/ticket/18465#comment:28>
WordPress Trac <http://core.trac.wordpress.org/>
WordPress blogging software

More information about the wp-trac mailing list