[wp-hackers] WordPress plugin inspections

Thu Feb 20 17:49:30 UTC 2014

Hi Chris,

I've agreed previously on this thread that "Unsafe to use" was too 
categorical, and have changed that text to "Potentially unsafe". We'll 
continue to make changes to ensure we're being as clear and useful as we 
can.

On the rest of your post, we may have to agree to disagree. It is not 
commercially viable for us, or anyone else, to do comprehensive code 
reviews for free. It is not acceptable, in my opinion, that people with 
serious WordPress sites so frequently install plugins with no idea as to 
their quality or security. We're trying to help by giving people enough 
information to make slightly better decisions - for example, by focusing 
the resources they do have on more thorough examination of candidate 
plugins that are most likely to be problematic.

I hope it's clear by now that I am committed to making this site and 
process better, and I'm very happy to take criticism and feedback that 
helps us to improve. But I think, for now, for this thread, I won't 
respond to more posts that say -- more or less -- "just don't do this". 
Because I think that what we're doing does more good than harm, even 
with its imperfections.

Cheers,

Harry

On 20/02/2014 17:41, Chip Bennett wrote:
> Again: you're announcing that the neighbor's shed *should be condemned*
> ("unsafe to use"), based on "indications of badness, but no specific
> vulnerabilities".
>
> That is precisely where I have a problem with what you're doing.
>
>
> On Thu, Feb 20, 2014 at 12:24 PM, Harry Metcalfe <harry at dxw.com> wrote:
>
>> Hi John,
>>
>> This - more or less - is exactly how we operate.
>>
>> We have a look. If we see indications of badness, but no specific
>> vulnerabilities, we write that up and publish the inspection.
>>
>> If we see vulnerabilities, we write up an advisory and disclose it
>> responsibly, exactly as you suggest (details: https://security.dxw.com/
>> disclosure/).
>>
>> I don't think it is necessary to disclose in advance for an inspection,
>> because we're not announcing that the neighbour's shed is broken. We're
>> announcing that neighbour's shed's looking a bit old and tatty, and that
>> people might not want to keep their stuff in it until it's fixed.
>>
>> Quite a few people have suggested that we should reach out to plugin
>> authors, though. I am, in principle, happy to do that. But such a mechanism
>> would have to be at least partly automated, and we have no private contact
>> details for plugin authors. So, the best we could do is probably to have a
>> bot that posts on people's forums. But that's more notification than
>> notice, and I'm not sure I'm comfortable with the idea of such a bot in any
>> event.
>>
>> If you have an idea for how we can reliably, semi-automatically give
>> authors notice, and then publish after some predefined time - I'm all ears.
>>
>> Harry
>>
>>
>>
>> On 20/02/2014 16:50, John wrote:
>>
>>> The community would be better served if you first contacted plugin authors
>>> and the maintainers of the WP plugin repo regarding security issues.
>>>
>>> If the door on your neighbor's shed was broken, making it easy for thieves
>>> to enter, would you first announce it to the whole community in a letter
>>> to
>>> the editor alongside an ad for your door repair services, or would you be
>>> Dudley Do-Right and tell your neighbor directly?
>>>
>>> If you've reviewed enough code to make the claims, you can certainly
>>> reveal
>>> specific vulnerabilities to the plugin authors and allow them to fix them.
>>> This is pretty much the way any open source community handles security
>>> issues. If you do enough of that, the money will come - if that's what you
>>> want.
>>>
>>> After a reasonable period of time after security updates have been
>>> released
>>> (or not in cases where plugin authors are unresponsive), the public
>>> service
>>> announcement could follow.
>>>
>>>
>>> On Thu, Feb 20, 2014 at 3:37 AM, Harry Metcalfe <harry at dxw.com> wrote:
>>>
>>>   Disappointingly, we'll perhaps have to agree to disagree.
>>>> I think the site is a positive contribution to WordPress's security.
>>>> Hopefully, in time, we'll earn some trust. I'm not expecting that to be
>>>> instant. I don't think we're condemning anybody: we're pointing out
>>>> issues
>>>> which are widely accepted to be indicative of problematic code.
>>>>
>>>> In the mean time, people are - of course - free to vote with their feet
>>>> and not visit the site. Or set up a better one.
>>>>
>>>> Harry
>>>>
>>>>
>>>> On 20/02/2014 01:05, Chris Williams wrote:
>>>>
>>>>   Let's see if I can summarize: you are using arbitrary criteria
>>>>> administered by people of unknown skill/experience and using the results
>>>>> to publicly condemn other people's work with an overly broad brush, and
>>>>> without any mechanism for recourse.  The result has no positive
>>>>> benefits.
>>>>> It demeans the plugin authors and their work, and by reflection your
>>>>> firm
>>>>> and its work, raises alarm in the community you claim to support, and
>>>>> garners you no goodwill.
>>>>>
>>>>> I'm sorry, but given the train wreck this has become, my best advice is
>>>>> precisely that: stop doing it.
>>>>>
>>>>>
>>>>> On 2/19/14 1:32 PM, "Harry Metcalfe" <harry at dxw.com> wrote:
>>>>>
>>>>>    But I do value the points you've made
>>>>>
>>>>>> and we will make some changes based upon then. I'd be keen to hear any
>>>>>> other feedback you might have later (short of "stop doing it"!)
>>>>>>
>>>>>>   _______________________________________________
>>>>> wp-hackers mailing list
>>>>> wp-hackers at lists.automattic.com
>>>>> http://lists.automattic.com/mailman/listinfo/wp-hackers
>>>>>
>>>>>   _______________________________________________
>>>> wp-hackers mailing list
>>>> wp-hackers at lists.automattic.com
>>>> http://lists.automattic.com/mailman/listinfo/wp-hackers
>>>>
>>>>   _______________________________________________
>>> wp-hackers mailing list
>>> wp-hackers at lists.automattic.com
>>> http://lists.automattic.com/mailman/listinfo/wp-hackers
>>>
>> _______________________________________________
>> wp-hackers mailing list
>> wp-hackers at lists.automattic.com
>> http://lists.automattic.com/mailman/listinfo/wp-hackers
>>
> _______________________________________________
> wp-hackers mailing list
> wp-hackers at lists.automattic.com
> http://lists.automattic.com/mailman/listinfo/wp-hackers