[wp-trac] [WordPress Trac] #53016: Robots API

[WordPress Trac]

#53016: Robots API
-----------------------------------------------+----------------------
 Reporter:  kraevit                            |       Owner:  (none)
     Type:  defect (bug)                       |      Status:  closed
 Priority:  normal                             |   Milestone:
Component:  Sitemaps                           |     Version:  5.7
 Severity:  normal                             |  Resolution:  invalid
 Keywords:  has-screenshots reporter-feedback  |     Focuses:
-----------------------------------------------+----------------------
Changes (by dd32):

 * keywords:  has-screenshots needs-testing => has-screenshots reporter-
     feedback
 * status:  new => closed
 * resolution:   => invalid
 * milestone:  Awaiting Review =>


Old description:

> Hello,
> My name is Krasimir Kraev, I have frustrating issue with my wordpress
> installation version 5.7 using yoast seo plugin. My google search console
> reported that i have 1600 invalid url's due to having "noindex" in the
> <meta robots tag> of my pages/posts. Everything in the admin panel is set
> correctly(eg settings reading). I found that in this particular file of
> wordpress core files - /wp-includes/robots-template.php in the lines from
> 153-165 there is a function:
> function wp_robots_no_robots( array $robots ) {
>
>     // $robots['noindex'] = true;  THIS DEFAULT LINE COUSING ISSUES
> INDEXING MY PAGES ON GOOGLE SEARCH CONSOLE so i changed to false
>     $robots['noindex'] = false;
>
>         if ( get_option( 'blog_public' ) ) {
>                 $robots['follow'] = true;
>         } else {
>                 $robots['nofollow'] = true;
>         }
>
>         return $robots;
> }
>
> Is this a bug or it is a normal wordpress 5.7 behavior?
>
> Please respond.
>
> website: https://webnavigator.bg
> template: Listify 2.13.7
> yoast seo 16.1.1
>
> Regards!

New description:

 Hello,
 My name is Krasimir Kraev, I have frustrating issue with my wordpress
 installation version 5.7 using yoast seo plugin. My google search console
 reported that i have 1600 invalid url's due to having "noindex" in the
 <meta robots tag> of my pages/posts. Everything in the admin panel is set
 correctly(eg settings reading). I found that in this particular file of
 wordpress core files - /wp-includes/robots-template.php in the lines from
 153-165 there is a function:
 {{{
 function wp_robots_no_robots( array $robots ) {

     // $robots['noindex'] = true;  THIS DEFAULT LINE COUSING ISSUES
 INDEXING MY PAGES ON GOOGLE SEARCH CONSOLE so i changed to false
     $robots['noindex'] = false;

         if ( get_option( 'blog_public' ) ) {
                 $robots['follow'] = true;
         } else {
                 $robots['nofollow'] = true;
         }

         return $robots;
 }
 }}}

 Is this a bug or it is a normal wordpress 5.7 behavior?

 Please respond.

 website: https://webnavigator.bg
 template: Listify 2.13.7
 yoast seo 16.1.1

 Regards!

--

Comment:

 Hi @kraevit,

 The function in question should only be called when the blog visibility
 option is enabled, or another plugin is causing it to be called.

 The functionality is controlled through the following setting under
 `Settings -> Reading`:
 {{{
 Search engine visibility        Discourage search engines from indexing
 this site
 }}}

 If that option is disabled, you should check where the function is being
 called from (It's new in 5.7), as I suspect it'll be being called from a
 plugin. The only place that calls it in WordPress that would affect most
 urls is `wp_robots_noindex()`.

 Without knowing which URLs are affected or why it's being called, it's not
 possible to determine if this is a WordPress issue or not, but it doesn't
 appear that it's likely to be a core bug due to the newness of the
 functionality and that more people aren't affected by it.

 Unfortunately I'm closing this as `invalid` as there's no information
 which can be used to replicate it or point to the cause being WordPress
 itself.

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/53016#comment:1>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform