[wp-hackers] GSoC Project: Performance Pre-application

[Jacob Santos]

With regards, I apologize for the late response. To reply, Google does 
not allow groups. If we are both accepted, we'll just work on our 
separate solutions and hopefully at the completion, bits and pieces of 
the highest quality will be used from either project. Good luck and if 
you are accepted, I look forward to seeing what solution you provide and 
how you handled it.

The baseline doesn't matter. I say this, because the goal is to find 
where the current application is taking the most time and try to 
optimize those areas. It might be you optimize one area, only to find 
that a completely different area pops up to be time consuming. I would 
suggest you run, fix, rinse, and repeat and continue until you can no 
longer optimize the core any longer.

You can benchmark the database queries using PHP. There is a library for 
mysql (which so happens is what WordPress uses) and mysqli, and 
eventually PDO, called mysqlnd. If you compile mysql using that library 
instead of the libmysqld library, then you'll gain the additional 
benchmarking functions. Eh. If you find a tutorial on how to do this, 
then it could probably be added to the benchmark plugin. The goal should 
also be to track down poorly optimized SQL queries and fix those as well.

http://www.santosj.name/general/wordpress-google-summer-of-code-project/

My solution for the file includes is only to see if putting all of the 
functions in a few files will help with the speed of the bootstrap. 
Perhaps one day, a better MVC implementation will be used that will 
allow for including only what is needed during any page load. However, 
for now, whatever method would require a build process, so it would only 
be to prove or disprove that using one file is an advantage to WordPress 
Execution speed. I still think that doing so would be a disadvantage for 
contributors with having one massive file with many many lines of code.

Sorry, I thought PerformancePress name was original, but it appears I 
borrowed the idea from you, Mahmoud Al-Qudsi. Or at the very least, we 
thought of the same generic term. I like the idea and it would be cool 
to see how you were able to implement it.



Thomas Bukowski wrote:
> On Mar 22, 2008, at 10:58 AM, Mahmoud Al-Qudsi wrote:
>
>>> This framework can take a form similar to a lot of the code in
>>> wordpress-tests/wp-testlib. My primary concern here is to make
>>> performance tuning relatively trivial to do, and hence can be done
>>> regularly, perhaps as each release's development is drawing to a
>>> close. I'm not sure if wordpress's development process needs this; it
>>> was, if anything, just a neat idea.
>>
>> But you need a baseline. Assuming the goal of the project is to 
>> improve the *current* codebase and /not/ just to make sure it doesn't 
>> get any worse; then you'll need to first fix the code *then* 
>> continuously build/test/monitor its performance, no?
>
> The idea (which I did not express very well) was to have that 
> framework; then a set of 'definitive' benchmarks to run, which gives a 
> bunch of numbers. Let's run that on, say, the final 2.5 release. Would 
> that be a baseline to work from?
>
> Once the framework is there it is easier to quantify just how much 
> better (or worse) a change is, be it core patch or plugin. Then the 
> actual work can start being done, in a similar way that wp-testlib has 
> first to be written before wp-testcases could be written?
>
> I wouldn't be surprised if someone has already written some sort of 
> php performance testing 'suite' - that could readily be adapted, if 
> possible.
>
>>> It could also handle figuring out which sections of the codebase don't
>>> scale well; i.e., what happens when you comment on a post with 100
>>> other comments? 10000 other comments?
>>
>> With regards to your questions about *database*-scaling, the answer 
>> is quite simple: so long as you're using a modern RDBMS and you have 
>> decently-coded queries (i.e. no recursive nested queries, etc.) the 
>> only difference should be in the time it takes for the response to be 
>> sent to the PHP parser and for the PHP parser to circle through that 
>> many records in linear progression. I've seen it myself on my own 
>> blog, which has posts from 0 to 1000 comments.
>
> Eek, bad example; but maybe I'm chasing an empty category. I wonder, 
> the description on the gsoc2008 codex page (which I assume you 
> wrote?), what does "don't scale well" mean? I took it to mean load 
> scaling, i.e. what happens when there are thousands of this request 
> rather than dozens.
>
> Since you've run some well-hit wordpress blogs, you'd have a good idea 
> where to start from?
>
>>> I believe xdebug can output files that can be visualized with
>>> KCachegrind; then I'd love to spend some time optimizing the most
>>> obvious bottlenecks. Not sure if that is worth including in the scope
>>> of a GSoC project?
>>>
>>> I think the idea of having a 'build' system that combines files
>>> together is worth having; opening the hundred-odd files that wordpress
>>> includes must be quite expensive? This is obviously a question that
>>> can easily be answered by testing with the framework.
>>
>> More importantly: *WHY* are we opening/including all those millions 
>> of files when we only a minor fraction of them to do what we 
>> currently need? It'd be better to (RE: the codex) only include/merge 
>> files needed for each single operation and then call that 
>> (operation,file) tuple when requests are made.
>
> That's an excellent idea. If there was some way to combine those files 
> before deployment, in a way that keeps wordpress's architecture 
> relatively ummodified for plugin compatibility, and also maintain 
> APC/etc opcode caching compatibility? It sounds like a tall order.
>
>> <Slightly-OT>
>> I just posted about the Performance project for WordPress GSoC 2008 
>> on my blog; not much details, but if anyone is interested:
>> http://neosmart.net/blog/2008/wordpress-performancepress-and-gsoc-2008/
>
> Are you still interested in releasing PerformancePress? Or, not 
> necessarily 'releasing' it, but making its code available in some form 
> so interested parties (e.x. me) could pursue and find out what you've 
> done differently?
>
> Thanks for your comments!
> nd.
> _______________________________________________
> wp-hackers mailing list
> wp-hackers at lists.automattic.com
> http://lists.automattic.com/mailman/listinfo/wp-hackers


-- 

Jacob Santos

http://www.santosj.name - blog
http://funcdoc.wordpress.com - WordPress Documentation Blog/Guide Licensed under GPLv2

Also known as darkdragon and santosj on WP trac.