[wp-trac] [WordPress Trac] #57625: WP_Query cache memory leak

[WordPress Trac]

#57625: WP_Query cache memory leak
--------------------------+------------------------------
 Reporter:  owi           |       Owner:  (none)
     Type:  defect (bug)  |      Status:  new
 Priority:  normal        |   Milestone:  Awaiting Review
Component:  Cache API     |     Version:
 Severity:  normal        |  Resolution:
 Keywords:                |     Focuses:  performance
--------------------------+------------------------------

Comment (by owi):

 Thank you for your insight @peterwilsoncc !

 > * a configuration error on the persistent cache: I think setting a TTL
 or `maxmemory` and an eviction policy would fix OOM errors

 I completely agree the need for eviction policy for edge cases where
 server is reaching max  memory - and that's a simple config setting. Redis
 (or other provider) should not choke and act before it gets OOM.

 But that is just a cure for the symptom and not the cause. Why should we
 be evicting potentially valid keys just because one of our cache groups is
 misbehaving and causing storage to operate constantly on the high (near
 max) mem usage?

 On a high traffic sites the ratio of wp_query cache keys versus other keys
 is like 10:1 just after 3-5 days. That's definitely unhealthy situation
 which should not happen. Turning on shorter TTLs or eviction policies for
 the whole server is IMO counterproductive for the caching in general.

 I went around it by developing a small solution for setting short TTL for
 the keys with the `wp_query` (posts group) and `get_term` (terms group)
 prefixes. Nevertheless as written in the ticket description I believe
 setting arbitrary number is still a sub-optimal thing though works in the
 short term.

 Alternatively I was thinking about writing a cron job to check
 `last_updated` key and remove the keys which are invalid (and thats
 probably the way to go)

 > The latter is difficult to resolve as not all persistent caches allow
 for items to be deleted by group or cache prefix and WordPress needs to
 account for that situation.

 That was my thinking - is there a possibility that query caches belong to
 its own cache group and we flush it on certain events? Or even better by
 setting an event in cron as it might be tricky/countrproductive doing that
 during regular operation? Is moving keys between cache groups backwards
 compatible in the WordPress context?

 While not all providers support flushing by prefix or by the group - I
 believe we still should handle that in some way for the ones who do. As
 all providers will suffer from that issue and we should reduce the impact
 of that problem.

 I am in a good position where I have full control over my redis servers
 and pretty advanced infrastructure. Many people might get a memcached or
 redis instance and have problems with their provider complaining about
 storage/memory use.


 Let me know if I can be of any help!

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/57625#comment:5>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform