[wp-trac] [WordPress Trac] #52900: Instantly index WordPress web sites content in Search Engines

[WordPress Trac]

#52900: Instantly index WordPress web sites content in Search Engines
-------------------------------------------------+-------------------------
 Reporter:  fabricecanel                         |       Owner:  (none)
     Type:  feature request                      |      Status:  new
 Priority:  normal                               |   Milestone:  Awaiting
                                                 |  Review
Component:  General                              |     Version:
 Severity:  normal                               |  Resolution:
 Keywords:  reporter-feedback has-patch has-     |     Focuses:
  unit-tests                                     |
-------------------------------------------------+-------------------------

Comment (by dd32):

 Replying to [comment:21 fabricecanel]:
 > We did our first pull request

 Hi @fabricecanel,

 To follow up on some earlier comments here - have you looked into
 integrating with either http://pingomatic.com/ or http://blo.gs/cloud.php
 ?

 They're admittedly not very modern API's, but benefit from millions of
 existing sites already making use of them, combined with existing
 standards such as Sitemaps it can provide what's needed without additional
 code on the clients side.

 There might also be room in the middle to act as a middleman - consuming
 those API's and relaying it onto Bing and others using the API, or having
 Pingomattic or blo.gs to relay it onwards to those too.

 Before this proposal is really viable to consider for WordPress inclusion
 (IMHO) there needs to be industry support on it being a generalised system
 that allows for all players (small and large) to be supported without
 additional need from site authors or software vendors. A standard is only
 truely open if multiple vendors support it, otherwise it's just an
 proprietary format that so happens to be documented publicly.

 To me, it seems that having client websites actively "pinging" select
 search engines added in WordPress core is not exactly open, I would want
 anyone interested in the data being able to access a stream of the changes
 - and having them get their crawler added to WordPress seems like a high
 barrier to entry.

 This seems like one of the major benefits of centralised open relay
 services like those mentioned above.

 I'm assuming that one of the reasons for this approach, based on the
 inclusion of a per-site key that can be validated through a HTTP callback,
 is that the existing methods include a lot of spam and lack of any way to
 verify that whom sent the request is actually the author of it. Monitoring
 the Blo.gs feed definitely shows a LOT of spam. While the key verification
 will allow verifying it is who they say they are, it won't prevent spam
 being pushed into the system.

 ----

 To throw some ideas in here:
  - What would need to be done to improve the existing pingback services in
 place?
  - Do they ''need'' to be replaced?
  - Do they need to supply extra details to clients to improve the service?

 Looking at the output from blo.gs feed:
 {{{
 <weblog name="My Site" url="https://example.org/" service="ping"
 ts="20210928T08:00:00Z" />
 }}}

 That's not super useful as-is, it doesn't say what changed, but the
 addition of a link to a) The sitemap and b) the page changed would benefit
 greatly and provide a lot of what this proposal adds.

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/52900#comment:22>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform