[wp-trac] [WordPress Trac] #18465: robots.txt should tell Google to not index wp-admin and wp-includes

[WordPress Trac]

#18465: robots.txt should tell Google to not index wp-admin and wp-includes
--------------------------+---------------------
 Reporter:  Viper007Bond  |       Owner:  ryan
     Type:  enhancement   |      Status:  closed
 Priority:  lowest        |   Milestone:  3.3
Component:  General       |     Version:  3.2.1
 Severity:  trivial       |  Resolution:  fixed
 Keywords:  has-patch     |
--------------------------+---------------------

Comment (by SergeyBiryukov):

 Replying to [comment:23 nacin]:
 > How will this work in a network situation?

 On a sub-domain install, any site would return this for a `robots.txt`
 request:
 {{{
 User-agent: *
 Disallow: /wp-admin/
 Disallow: /wp-includes/
 }}}
 On a sub-directory install, `robots.txt` (with the same content) is only
 served for the main site.

 Replying to [comment:24 scribu]:
 > I guess we should use a leading wildcard, like it was suggested in
 [#comment:10].

 Since wildcards in Disallow statements are not officially supported by the
 protocol, I guess we should leave it as is for now.

-- 
Ticket URL: <http://core.trac.wordpress.org/ticket/18465#comment:25>
WordPress Trac <http://core.trac.wordpress.org/>
WordPress blogging software