[wp-hackers] GSoC Proposal: Integrate WP-cache / WP Super Cache into WordPress

[Computer Guru]

I've worked extensively with WordPress caching projects (both HTML and
object caches, having rewritten WP-Cache for IIS, helped Donncha with
a bunch of WP-SuperCache stuff, and written caching engines for XCache
and eAccelerator) and I've come to one really, really scary
conclusion: all they do is make up for the original WP bloat.

And, believe me, I don't say this lightly. I was getting real fed-up
with WordPress dying under joint Slashdot-Digg traffic no matter what
caching solution I used on a dedicated host so I decided to rewrite
the main page and the single-post pages in plain PHP from scratch. I
did - plain PHP & HTML with some simple EZSQL functions, hard-coded
themes, etc. and the difference was unbelievable.

See for yourself:
http://neosmart.net/blog/
vs
http://neosmart.net/blog/page/2/
(page load times at the bottom, current iteration *does* make use of
*object* caching, no HTML cache)

Whereas WP would take 5+ seconds under a *full* load to create a
single page (and far, far too many queries), the rewritten frontend
never went above .01 seconds/page on average at the peak of the
digg-slashdotting effects.

Bottom-line: Caching, in all its forms, is a way to get that last bit
of performance boost to make your app a real killer. It is NOT a way
to make up for badly-performing code in the first place.

I understand the difficulties of making flexible-yet-well-performing
code. I know the tradeoffs between hard-coding stuff and having good
page-load-times. But at the end of the day, /caching is NOT
WordPress's biggest problem/, rather loading too many files at once,
using way more nested loops than it should, and not skipping certain
sections of the code when it can are what's responsible for WP's
"legendary" fear of the /. effect.

A well-written package doesn't need caching of any sort or form, just
a simple opcode cacher in the backend should more than suffice. The
replacement frontend for WP (which I'm considering releasing) was
generating pages in 0.00xx seconds, w/ object caching it does it in
0.000x seconds instead (warm hits, that is). I had attempted to
object-cache everything I could see in a stock WP install, but that
didn't give me anywhere near the performance boost I was looking for.

I'm willing to mentor any performance-related projects for WP.

On 3/1/08, Andy Skelton <skeltoac at gmail.com> wrote:
> On Fri, Feb 29, 2008 at 12:17 AM, Ronald Heft <ron at cavemonkey50.com> wrote:
>  >  excellent opportunity to start fresh with a new caching system. If anyone
>  >  has any examples of a PHP-based caching system in they feel is better than
>  >  WP Super Cache, I would love to see it.
>
>
> I've done a lot of work on the caching systems that WordPress.com
>  uses, severely hacking on Ryan's memcached client, and I wrote the
>  HTML cache that helps burst-traffic event blogs like live.gizmodo.com
>  stay up when most others fail.
>
>  I don't know of anything better than Donncha's Super Cache for what it
>  does. My HTML cache (I named it supercache before Donncha released his
>  Super Cache, but I never released mine so he won the name recognition
>  in the end) is based on memcached and that makes it a poor fit for the
>  mainstream right now. That's not because memcached is hard to set up
>  (it's not) or hard to code for (we have working clients) but shared
>  hosting setups typically don't allow it.
>
>  Some form of worthwhile persistent caching should work in the default
>  install with the minimum requirements. Obviously we aren't there yet.
>  If we keep the requirements as they are, we will have to put the cache
>  in the database. Caching in the database would only be worthwhile if
>  we cached very high-level objects, the highest level being fully
>  rendered pages. Obviously this would not be as good as an APC object
>  cache, but for the minimum install it could do wonders.
>
>  What I learned from my experiences with WordPress and caching is that
>  there are more variables than you can hope to account for. (You can
>  take the WordPress out of the dynamic but you can't take the dynamic
>  out of WordPress?) You can get a lot of mileage from caching at any
>  level (queries, values, objects, rendered HTML elements, entire pages)
>  but the higher up that stack you go, the harder it gets because there
>  are so many factors contributing to the rendered page.
>
>  Back to my supercache: it's painfully simple and extremely effective
>  at serving a page to new visitors. It allows pages to be freshly
>  rendered for cookie holders or whatever criteria you care to configure
>  for identifying visitors who require it. The whole thing is about 200
>  lines of PHP including comments. Using a similar test for cache
>  eligibility, a db-backed HTML cache could extend the operational
>  availability of sites that experience surges of new visitors to single
>  posts (a typical Digg scenario). This would have to be significantly
>  more complex than my supercache because we would have to code even the
>  basic mechanism of expiration and cleanup into the cache. However, if
>  we are only caching entire pages, this can be pretty easy.
>
>  One thing that kills an HTML cache is too-dynamic content. Random
>  output (e.g. random-ordered blogroll), time-dependent elements (e.g.
>  clocks), and HTTP header-dependent elements (e.g. seach term
>  highlighting [Referer] or Firefox buttons [User-Agent]) all become
>  frozen in the cache and should not be used on blogs that serve
>  pre-generated pages unless they are generated client-side with JS,
>  Flash, etc., or the cache is set up to vary accordingly.
>
>  Wrapping up, I think there should always be attention devoted to
>  improving WordPress resilience on the minimum setup and I would like
>  to see this as a GSoC project.
>
>  Cheers,
>
> Andy
>
> _______________________________________________
>  wp-hackers mailing list
>  wp-hackers at lists.automattic.com
>  http://lists.automattic.com/mailman/listinfo/wp-hackers
>


-- 
Computer Guru
Director,
NeoSmart Technologies
http://neosmart.net/blog/