[wp-meta] [Making WordPress.org] #4184: Add a robots.txt disallow all rule to ps.w.org

[Making WordPress.org]

#4184: Add a robots.txt disallow all rule to ps.w.org
----------------------------+-----------------------
 Reporter:  jonoaldersonwp  |       Owner:  Otto42
     Type:  defect          |      Status:  accepted
 Priority:  lowest          |   Milestone:
Component:  General         |  Resolution:
 Keywords:  seo             |
----------------------------+-----------------------

Comment (by dd32):

 Committing a file won't work here, as it's the robots.txt served by apache
 for all SVN hosts, there's a similar generic one for Trac hosts.

 @jonoaldersonwp can you confirm what should happen for routes such as
 these, and if they should differ from https://ps.w.org/robots.txt (That's
 just a cached CDN of plugins.svn.wordpress.org)
 http://plugins.svn.wordpress.org/robots.txt
 http://core.svn.wordpress.org/robots.txt
 http://meta.svn.wordpress.org/robots.txt

 Having them have different content is possible, it's just easier to have a
 single file for all svns.

-- 
Ticket URL: <https://meta.trac.wordpress.org/ticket/4184#comment:2>
Making WordPress.org <https://meta.trac.wordpress.org/>
Making WordPress.org