[wp-hackers] should (can?) wp-cache be adopted into the core?

Wed Apr 18 15:03:27 GMT 2007

Jamie, that's really weird.

For the month of March, I was averaging 375k hits a day on my one server, on
a single-core P4 2.8GHz, with about 260k of those going to WP, the rest to
various DB-intensive things like Gallery2 and the forum.
(http://neosmart.net/ && http://neosmart.net/blog/) 

This is just a single server, tons of plugins, MySQL, IIS, URL Rewriters, no
AJAX, etc.

Like I said in my previous email, I'm not using any SQL-caching techniques. 
Disable unneeded/unused PHP extensions (or if on *nix, recompile without
them). Are you using APC/eAccelerator?

You shouldn't need 8 cores for < 400k hits/day, esp. seeing as you're
offloading comments to HS.

If you want to discuss this further, feel free to hit me off-list.

Computer Guru
NeoSmart Technologies
http://neosmart.net/blog/

> -----Original Message-----
> From: wp-hackers-bounces at lists.automattic.com [mailto:wp-hackers-
> bounces at lists.automattic.com] On Behalf Of Jamie Holly
> Sent: Wednesday, April 18, 2007 4:33 PM
> To: wp-hackers at lists.automattic.com
> Subject: RE: [wp-hackers] should (can?) wp-cache be adopted into the
> core?
> 
> I agree 1000%. I do the tech work on two sites. One is averaging
> 250,000
> hits a day and the other about 120,000 hits a day. Without running wp-
> cache,
> our 8 core server dies. We also aren't running that many plugins, and
> the
> ones we are have been run through the mill on profiling and not causing
> any
> significant load. We also have a small shell script that runs, which
> downloads the front page and generates an index.html for it to keep
> loads
> down (which is a must for us), as well as using feedburner to handle
> the
> additional 100,000+ feed subscribers for both sites.
> 
> One thing I think would help out a lot is to re-examine the filters.
> The
> larger blog, we are using Haloscan. The smaller one is using WP-
> Comments,
> and that one gets anywhere from 200-600 comments per post (with 8-10
> posts
> per day). On that site I changed the comments so they run off AJAX to
> help
> with the load. I have a refresh comments script that reloads the newer
> comments (which does not load the wp-core, but rather contains all the
> filter functions within the single file and generates the output). It
> also
> does comment posting via AJAX, but I still load the core in that to
> handle
> posting better.
> 
> With all that said, it seems that we might be wasting a lot on comment
> filtering. For every comment on every page we look at these filters
> being
> loaded:
> 
> add_filter('comment_author', 'wptexturize');
> add_filter('comment_author', 'convert_chars');
> add_filter('comment_author', 'wp_specialchars');
> 
> add_filter('comment_email', 'antispambot');
> 
> add_filter('comment_flood_filter', 'wp_throttle_comment_flood', 10, 3);
> 
> add_filter('comment_url', 'clean_url');
> 
> add_filter('comment_text', 'convert_chars');
> add_filter('comment_text', 'make_clickable', 9);
> add_filter('comment_text', 'force_balance_tags', 25);
> add_filter('comment_text', 'wpautop', 30);
> add_filter('comment_text', 'convert_smilies', 20);
> 
> I have been toying with ideas in my mind to call those before the
> comment is
> saved and then don't call it when the page is viewed. The big problem
> is
> that we have close to 1,000,000 comments in the database right now, so
> I
> need to come up with a way for Wordpress to tell if the comment has
> been
> filtered (possible Boolean added to each comment - is_filtered), and
> then
> using a cron job to filter out X amount of comments every Y minutes
> until
> everything is caught up.
> 
> I did see a ticket awhile back for the same filter rearrangement to
> take
> place on posts also and it seemed to be a very popular idea.
> 
> Something else I saw Matt mention in this thread was making WP more
> modular
> to that it only loads what is needed versus the entire core. This is
> something else that can help out a lot and I would like to see plugin
> authors take the same approach. Some plugins require a lot of coding
> for the
> admin code then very little for the actual page generation code. Having
> PHP
> load and compile that admin code on every page view is a waste. It
> would be
> better to move plugins to their own directory and do something as
> simple as
> having the main plugin file load then if the user is on an admin page,
> load
> the file containing all admin related code.
> 
> 
> 
> Jamie Holly
> http://www.intoxination.net
> > -----Original Message-----
> > From: wp-hackers-bounces at lists.automattic.com [mailto:wp-hackers-
> > bounces at lists.automattic.com] On Behalf Of Viper007Bond
> > Sent: Wednesday, April 18, 2007 8:37 AM
> > To: wp-hackers at lists.automattic.com
> > Subject: Re: [wp-hackers] should (can?) wp-cache be adopted into the
> > core?
> >
> > Couldn't agree more. I'd love to see a output caching system built in
> > (or
> > even better, as a bundled plugin to make it modular). While WordPress
> > can
> > run great under most circumstances, some blogs are just too dang
> > popular to
> > run without a super computer. Having something that is essentially
> > core, but
> > easily loadable / unloadable (a plugin) would solve this problem and
> > make
> > WordPress even more feasible / desirable for large bloggers.
> >
> > Case in point: I have a WP powered site that gets around a million
> hits
> > a
> > month. While I know I could run it without WP-Cache, I'd rather put
> the
> > CPU
> > to better use (like handling the site's forums which get around
> another
> > million or two hits a month). Not having to hack WP-Cache to work and
> > knowing for sure it'd support future versions would be a godsend.
> >
> > On 4/17/07, Matt Mullenweg <m at mullenweg.com> wrote:
> > >
> > > David Chait wrote:
> > > > Opinions?  Matt?  (since I think staticize was your baby
> > originally,
> > > no?)
> > >
> > > Having a built-in output cache could be very powerful for the
> growing
> > > number of WordPress users who are climbing to the top of the
> > blogosphere.
> > >
> > > There are a lot of issues to work out, as have been mentioned on
> this
> > > thread, so this is a good example of a project that needs a
> champion
> > to
> > > take ownership of the issue and coordinate what needs to happen to
> > get
> > > the plugin or functionality into a release like 2.3 or 2.4.
> > >
> > > --
> > > Matt Mullenweg
> > >   http://photomatt.net | http://wordpress.org
> > > http://automattic.com | http://akismet.com
> > > _______________________________________________
> > > wp-hackers mailing list
> > > wp-hackers at lists.automattic.com
> > > http://lists.automattic.com/mailman/listinfo/wp-hackers
> > >
> >
> >
> >
> > --
> > Viper007Bond | http://www.viper007bond.com/
> > _______________________________________________
> > wp-hackers mailing list
> > wp-hackers at lists.automattic.com
> > http://lists.automattic.com/mailman/listinfo/wp-hackers
> 
> _______________________________________________
> wp-hackers mailing list
> wp-hackers at lists.automattic.com
> http://lists.automattic.com/mailman/listinfo/wp-hackers