[wp-hackers] Help with the API on WordPress.org?

[Mike Schinkel]

DD32 <wordpress at dd32.id.au> wrote:
> * I was going to do the *exact* same thing as you are doing now, But...

Ah, I was curious why it wasn't that way because I know you guys are good
enough to do so... :-)

> i quickly stoped, for 1 reason, Google doesnt like Screen scrapers, Google
block you off after 2-3 similar repeditive searches, do it enough times
(like the plugin search) and you'll get blocked very quickly, Google's good
at detecting this.

Good point, but here are my thoughts on that. If you were doing for
WordPress.org I'd agree, but if doing in a plugin where the user
will probably only do a few searches at any one time I think they shouldn't
care so much and that they would have a very hard time to detect. If it
becomes a problem I could simulate the user's browser using their user
agent, etc. Since this would be such a low volume thing on a per person
basis I can't see it causing that much trouble?

Also, I'm doing it for my own use but planning to make available for others
who want to use it.

> So, Lets use one of the available Google API's! ...Then you realise the
 only ones available only return the first 10 results with no further
listing..

> Actually today you can get 4 or 8 results and then you can get subsequent
> pages. BUT, the showstopper for me was their lack of "exclude"
> functionality. Sure I could do with PHP but then I have to add in a JSON
> library and I decide it would just be easier to parse the HTML.
>
> Also, mdawaffe was looking into fixing up the .org searching using a
proper search engine functionality (Currently bbPress's searching just
doesnt cut
it), I'm not sure if its been released yet, But i believe its in the works.
> So for the time being, I'd advise you to not waste your time on a Google search
option, Yahoo might be possible, They actually have API's they like
you using.. and since .org will hopefully get a better search functionality
in the near future, It might not be worth your time (& the extra requests
which would need to happen for it)

> I have a big project coming up next week where I am going to be doing LOTS
> of plugin research so I've got to get this working for my own self ASAP. :-)
> I'll look into Yahoo but I've got Google working for now and the real
> problem is the WordPress.org's search and/or API.
>
> However, Since its not that complex, There isnt a function to send a list
of
slugs and get a list of details back, Only a function to return those
details for a single plugin -- Mainly because there was no need for that
kind of functionality in the requirements at the time.. :)

> Ugh :-(   I guess I could scrape WordPress.org and cache locally. Yuck.
>
> Any chance I could write an alpha version that could be hosted on
> WordPress.org?
>
> BTW, even though it's now what I needed thanks for documenting the API!
>
-Mike Schinkel
http://mikeschinkel.com