[wp-hackers] Proposal for a function commenting convention

[Kimmo Suominen]

Thanks for the clarification -- I think it makes the numbers more
useful, as I can understand what they represent.  :)

On Sun, Oct 14, 2007 at 01:18:11AM -0700, Travis Snoozy wrote:
> On Sun, 14 Oct 2007 10:31:25 +0300, Kimmo Suominen
> <kimmo at global-wire.fi> wrote:
> 
> > Hi Travis,
> > 
> > Numbers are good, but I'm not sure you are using the right ones.  The
> > real time is wall clock time, and is affected by all jobs running on
> > the system.  I think the relevant number is the sum of user and sys
> > times.
> 
> The system was very lightly loaded. I locked the CPU to its maximum
> frequency for the tests, and it idles at ~2-3% there. Wall clock time
> was used because it was the only thing marginally comparable to the page
> generation time I'd see in the embedded WordPress page generation stats.

The page generation stats shown by WP (timer_start() and timer_stop())
are indeed wall clock time as well.

However, it still feels wrong to use wall clock time to measure the
impact of comments in the code.  There are more factors that affect
the amount of wall clock time passing.

On the other hand, the sys time should account for the increased time
to read the larger source file, and the user time should account for
the increase in processing the comments.

> <snip>
> > To get the impact on PHP execution time, you should run PHP directly
> > from the command line using the time command.  Maybe this is what
> > you did, but it is not clear to me from your message.
> 
> I ran the test load from the command line. The WordPress home page for
> my test installation was generated via Apache; the generation time was
> taken from the embedded generation stats at the end of the page. These
> values should not be affected by Apache (AFAIK).

This is again going near the outer edges of my knowledge.  But I think
that when using mod_php, other things happening in Apache can have an
impact.  For example, I'm not sure what things have mutexes, if any,
that may need to be acquired by the thread executing the PHP code in
question, and if those can be contested by other parts of Apache.

> > Even then you should probably run something like 1000 repeats of
> > each, and use the average (or median or something -- I'm by no means
> > well versed in statistics).
> 
> "Statistically significant" tends to start at 20, but I didn't care
> *that* much ;). Standard deviation et. al. would be nice for getting a
> higher confidence in the numbers, but even taking the margin of error
> up to +/-50% would still leave 3.5% on the low end and 24% on the
> high.

An increased number of tests might reveal that the measurement is not
suitable for detecting a difference.  Maybe.  :)

Best regards,
+ Kimmo
-- 
<A HREF="http://kimmo.suominen.com/">Kimmo Suominen</A>