[wp-trac] [WordPress Trac] #47711: noindex, wpkill and admin-ajax.php

[WordPress Trac]

#47711: noindex, wpkill and admin-ajax.php
--------------------------------------+-----------------------------
 Reporter:  harryfear                 |      Owner:  (none)
     Type:  defect (bug)              |     Status:  new
 Priority:  normal                    |  Milestone:  Awaiting Review
Component:  General                   |    Version:  5.2.2
 Severity:  normal                    |   Keywords:
  Focuses:  administration, rest-api  |
--------------------------------------+-----------------------------
 Hi folks,

 I've found what appears to be a bug that affects fresh WordPress installs,
 so I am sure it's about the core and not themes or plugins.

 If one browses directly to "admin-ajax.php", the URL dies with code 400 as
 a result of the:

 {{{
 wp_die( '0', 400 );
 }}}

 Further down in the file, there is the noindex header:

 {{{
 @header( 'X-Robots-Tag: noindex' );
 }}}

 My issue arose due to Google reporting a "robots blocked but indexed"
 issue [https://webmasters.stackexchange.com/questions/123439/how-to-allow-
 only-admin-ajax-php-to-be-crawled].

 But when it dies it seems not to send the noindex header, at least on my
 hosting environment. I don't have any other issues with this host. My host
 is a basic shared Cpanel host with a bunch of different WordPress
 sites/domains running on it. I think this is something that could be
 looked at from the WordPress side as a possible bug?

 On some others' publicly-browsable sites, I can see that the noindex
 header *is* sent along with the 400 error status.

 I don't myself have access to multiple different hosting environments to
 test this with, so I'm throwing it out here for advice/expertise/support
 /alarm-raising.

 The noindex header instruction is set explicitly lower down in the admin-
 ajax.php file. If I copy that line up in way of a hot fix tweak on the
 code, the issue is resolved for me with Google:

 {{{
 // Require an action parameter
 if ( empty( $_REQUEST['action'] ) ) {
         @header( 'X-Robots-Tag: noindex' ); // I moved this up and it
 solves the issue
         wp_die( '0', 400 );
 }
 }}}

 I also contacted my host and we found that a .htaccess directive hot-fixes
 my issue on all my domains for now:

 {{{
 <FilesMatch "admin-ajax.php">
 Header set X-Robots-Tag "noindex"
 </FilesMatch>
 }}}

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/47711>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform