[wp-trac] [WordPress Trac] #33156: Allow admin-ajax crawling

[WordPress Trac]

#33156: Allow admin-ajax crawling
-------------------------+------------------------------
 Reporter:  joostdevalk  |       Owner:
     Type:  enhancement  |      Status:  new
 Priority:  normal       |   Milestone:  Awaiting Review
Component:  General      |     Version:
 Severity:  normal       |  Resolution:
 Keywords:  needs-patch  |     Focuses:
-------------------------+------------------------------

Comment (by dmchale):

 Replying to [comment:3 peterwilsoncc]:
 > For what it's worth, I'd rather allow everything in robots.txt and use
 noindex,nofollow meta tags for private sites & preventing indexing of wp-
 admin. Google recommends it as a more effective method for
 [https://support.google.com/webmasters/answer/6062608?hl=en preventing
 indexing].
 That was the solution I was alluding to in my comment to Joost above.
 [https://yoast.com/wordpress-robots-txt-example/#updates Back in
 February], he recommended getting rid of the /wp-admin block entirely. But
 I didn't want to create that patch without a conversation happening first,
 either, since that wasn't his suggestion as the OP on this ticket. Would
 be very easy though, we'd just have to remove everything in the "else"
 side of the $public check. A default file would still be returned, albeit
 nearly blank, and we still have the ability to write the `Disallow /` if
 the site isn't in public mode.

 Replying to [comment:3 peterwilsoncc]:
 > The WordPress coding standards are to use
 [https://make.wordpress.org/core/handbook/best-practices/coding-
 standards/php/#indentation tabs not spaces] for indention. Would you mind
 refreshing the patch?
 Thanks for the heads up. New install of PHPStorm on this pc, and I forgot
 to turn my whitespace highlighting on. Fixed now, shouldn't happen again.
 :) Since Mark already submitted one with tabs, I won't clutter things up
 with another copy.

--
Ticket URL: <https://core.trac.wordpress.org/ticket/33156#comment:4>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform