[wp-meta] [Making WordPress.org] #4068: Change 503 behaviour on paginated states

[Making WordPress.org]

#4068: Change 503 behaviour on paginated states
----------------------------+---------------------
 Reporter:  jonoaldersonwp  |       Owner:  (none)
     Type:  defect          |      Status:  new
 Priority:  high            |   Milestone:
Component:  General         |  Resolution:
 Keywords:  seo             |
----------------------------+---------------------
Description changed by SergeyBiryukov:

Old description:

> Requests for paginated states (i.e., /page/n/) by known bots are
> intentionally served a 503 HTTP status (and a ‘raw’ error template); this
> is managed by the load balancer.
>
> This is a critical issue for SEO, and must be altered as a matter of
> urgency.
>
> Full context, rationale, supporting information and details (including
> longer-term improvements) can be found here:
> https://docs.google.com/document/d/1smvevKgp29CP9mop7s8F-
> q8WhMo7cTipEWAqS2wehWI/edit?usp=sharing
>
> **Short-term actions:**
>
> - Reduce the maximum number of paginated results for logged-out users. At
> the moment, we stop pagination at page 100 (even if there are more
> results than this) and return a 404 error. We should reduce this
> threshold to ''30'' for logged-out users. Requests to page > 30 should
> return a 404 error.
>   - See https://meta.trac.wordpress.org/ticket/3985 for making the load
> balancer return a 'friendly' error page in these scenarios.
>   - We should add a notice at the end of page 30, indicating that people
> can search for more/different results if they haven’t found what they’re
> looking for.
>
> - We should increase the number of results on each page (currently 14) to
> 20 to offset the reduction in crawling/discovery.

New description:

 Requests for paginated states (i.e., /page/n/) by known bots are
 intentionally served a 503 HTTP status (and a ‘raw’ error template); this
 is managed by the load balancer.

 This is a critical issue for SEO, and must be altered as a matter of
 urgency.

 Full context, rationale, supporting information and details (including
 longer-term improvements) can be found here:
 https://docs.google.com/document/d/1smvevKgp29CP9mop7s8F-
 q8WhMo7cTipEWAqS2wehWI/edit?usp=sharing

 **Short-term actions:**

 - Reduce the maximum number of paginated results for logged-out users. At
 the moment, we stop pagination at page 100 (even if there are more results
 than this) and return a 404 error. We should reduce this threshold to
 ''30'' for logged-out users. Requests to page > 30 should return a 404
 error.
   - See #3985 for making the load balancer return a 'friendly' error page
 in these scenarios.
   - We should add a notice at the end of page 30, indicating that people
 can search for more/different results if they haven’t found what they’re
 looking for.

 - We should increase the number of results on each page (currently 14) to
 20 to offset the reduction in crawling/discovery.

--

-- 
Ticket URL: <https://meta.trac.wordpress.org/ticket/4068#comment:2>
Making WordPress.org <https://meta.trac.wordpress.org/>
Making WordPress.org