[wp-trac] [WordPress Trac] #46286: "Silence is golden" scenarios should return a 404 HTTP header

[WordPress Trac]

#46286: "Silence is golden" scenarios should return a 404 HTTP header
----------------------------+-----------------------------
 Reporter:  jonoaldersonwp  |      Owner:  (none)
     Type:  defect (bug)    |     Status:  new
 Priority:  normal          |  Milestone:  Awaiting Review
Component:  General         |    Version:
 Severity:  minor           |   Keywords:  seo analytics
  Focuses:                  |
----------------------------+-----------------------------
 Direct requests to system file structures/folders (e.g.,
 https://yoast.com/app/wp-content/)  currently return a blank page with a
 200 HTTP status. This informs search engines and agents that the
 page/state is 'okay', which may result in them being crawled, indexed, or
 otherwise exposed.

 This causes a number of SEO and analytics headaches, and, is generally
 poor practice.

 Whilst these directories/files technically ''do'' exist, and therefore
 arguably ''should'' return a 200 status, the important thing to consider
 is that ''no user-facing page/content exists at the URL''. We're not
 trying to represent the 'physical' file structure of the site (which we're
 already extremely abstracted from), we're trying to signpost what is, or
 isn't, a valid page/response. That an actual file/folder resides at the
 requested URL is irellevant.

 **In the short-term, we should simply set a 404 header in all 'Silence is
 golden' templates.**

 In the long-term, we should aim to find a way to allow all request to
 'Silence is golden' files to return the 404 template of the active theme.
 This may come with complexities, where we can't rely on system
 configurations/capabilities or folder structures to be consistent between
 sites.

 As an aspiration, we should aim that all requests to a WP site return an
 appropriate HTTP status and experience based on the user's context,
 regardless of the actual file/directory structure.

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/46286>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform