[wp-trac] [WordPress Trac] #52536: Add "noindex" to search results by default and "X-Robots-Tag: noindex" to feeds by default

[WordPress Trac]

#52536: Add "noindex" to search results by default and "X-Robots-Tag: noindex" to
feeds by default
-------------------------+-----------------------------
 Reporter:  pikamander2  |      Owner:  (none)
     Type:  enhancement  |     Status:  new
 Priority:  normal       |  Milestone:  Awaiting Review
Component:  General      |    Version:
 Severity:  major        |   Keywords:
  Focuses:               |
-------------------------+-----------------------------
 We’ve noticed that spammy websites are linking to RSS search results on
 our site that include their domain in the search terms.

 For example, if our site is wordpress.org and their site is example.com,
 they might link to
 wordpress.org/search/+example.com+best+pharmacy+pills+online/feed/rss2/

 For normal WordPress searches, this isn’t a problem because the search
 results are set to “noindex”. However, these RSS2 pages are outputted as
 XML and don’t include any kind of “noindex” tag, so Google recognizes them
 as being indexable pages.

 Looking around Google, it seems like this type of blackhat SEO technique
 is fairly common and most likely done in bulk by bots.

 SEO plugins like Yoast and AIOSEOP appear to add a "noindex" tag to the
 search result pages, but neither of them seems to add that response header
 to the feeds, which means that most WordPress sites are vulnerable to that
 tactic.

 Since feeds and search pages are built into WordPress's core and most
 people wouldn't want their results to be indexed anyway, can we add
 noindex to those two types of pages by default?

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/52536>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform