[wp-trac] [WordPress Trac] #50456: Multisite robots.txt files should reference all network XML sitemaps

[WordPress Trac]

#50456: Multisite robots.txt files should reference all network XML sitemaps
----------------------------+------------------------------
 Reporter:  jonoaldersonwp  |       Owner:  (none)
     Type:  defect (bug)    |      Status:  new
 Priority:  normal          |   Milestone:  Awaiting Review
Component:  Sitemaps        |     Version:
 Severity:  normal          |  Resolution:
 Keywords:  seo             |     Focuses:  multisite
----------------------------+------------------------------

Comment (by pbiron):

 After some investigation, even the "simple" approach mentioned by
 @swissspidy
 [https://wordpress.slack.com/archives/CTKTGNJJW/p1593527966162400 in
 slack], is unacceptably slow, even for moderately sized networks (e.g.,
 ~2000 sites):

 {{{
 #!php
 if ( ! wp_is_large_network() ) {
         foreach( get_sites() as $site_id ) {
                 echo "Sitemap: " . get_home_url( $site_id, '/wp-
 sitemap.xml' );
         }
 }
 }}}

 since `get_home_url()` does a `switch_to_blog()`...and it is the blog
 switching which slows things down.

 I've experimented with a couple of different approaches to use WP-Cron to
 write to the `blogmeta` table (or a transient on the main site), which
 will allow `WP_Sitemaps::add_robots()` to efficiently get the entries to
 be added to `robots.txt` without having to do `switch_to_blog()` during
 the generation of `robots.txt`.

 I think something like that is ultimately the way to go, but there's not
 time before 5.5 Beta 1 ships next week to get that correct.

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/50456#comment:12>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform