[wp-hackers] Re: [spam-stopper] New plugin version

[Sebastian Herp]

Matt Mullenweg wrote:
> I would posit that an independent plugin system like that would always 
> be less effective because none of the weightings would know anything 
> about the other weightings, and it would really only be useful to 
> people interested in endlessy tweaking the scoring. You'd have to test 
> every change in weighting against a non-trivial corpus of known spam 
> and ham. (At least a million of each.)
>
You are right. I even do this in the Spam Karma version of the Akismet 
plugin ... it skips this test if the other tests are fairly sure it's 
spam or ham. So the order and the weighting lead to the final decision, 
but getting both right could be too much for the ordinary user ...

> Also how would you weight an external system like Akismet, which 
> already does thousands of tests and is being constantly adjusted?
I'd weight Akismet very low, because it is external. It simply has not 
as much information about the comment as I (meaning my webserver) have, 
so it can only be a rough filter or a last test if other tests don't 
lead to a clear decision, but nothing perfect (Akismet still does not 
recognize a minimum of 2 spams a day on my blog).

>
> I don't like tests that sit on the fence. Either call it or don't.
>
A confidence level indication of that call would be nice though ;-)