[wp-trac] [WordPress Trac] #43590: Search Engine Visibility option does not work as intended

[WordPress Trac]

#43590: Search Engine Visibility option does not work as intended
----------------------------------------+-----------------------------
 Reporter:  mamaedler                   |       Owner:  donmhico
     Type:  defect (bug)                |      Status:  assigned
 Priority:  normal                      |   Milestone:  5.3
Component:  General                     |     Version:  2.1
 Severity:  normal                      |  Resolution:
 Keywords:  has-patch needs-unit-tests  |     Focuses:  administration
----------------------------------------+-----------------------------
Changes (by peterwilsoncc):

 * keywords:  has-patch => has-patch needs-unit-tests
 * owner:  (none) => donmhico
 * version:  4.9.4 => 2.1
 * status:  new => assigned
 * milestone:  Awaiting Review => 5.3


Comment:

 Thanks @jonoaldersonwp, following your feedback I've put this on the 5.3
 milestone.

 > That might result in, e.g., an assets folder in a plugin being crawled
 and indexed.

 * Do you think it's worth retaining the disallow rule for `wp-content` on
 private sites?
 * For Core's silence is golden files, such as `wp-content/index.php`, we
 can add `X-Robots-Tag` headers ''on another ticket''.

 @donmhico I've assigned this to you for following up modifying the meta
 tag to:

 * **retain `follow` on public/indexed sites** (so we don't break SEO and
 other plugins calling the function expecting pages to be followed)
 * modify the meta tag to **use `nofollow` on sites set to be private
 only**, per the recommendations in the comment above.

 Unit tests for the above change would be most helpful but I realise your a
 new contributor so may need some assistance with these.

 If you don't have time to work on this, feel free to unassign yourself.

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/43590#comment:11>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform