[wp-trac] [WordPress Trac] #60375: Site Transfer Protocol

[WordPress Trac]

#60375: Site Transfer Protocol
-------------------------+------------------------------
 Reporter:  zieladam     |       Owner:  (none)
     Type:  enhancement  |      Status:  new
 Priority:  normal       |   Milestone:  Awaiting Review
Component:  Import       |     Version:
 Severity:  normal       |  Resolution:
 Keywords:               |     Focuses:
-------------------------+------------------------------

Comment (by zieladam):

 @dmsnell I just realized we need to answer this fundamental question
 before going further:

 Is ''Site Transfer'' a direct Host <-> Host operation with optional
 support for .zip uploads? Or is it an export&download -> upload&import
 situation?

 I assumed this ticket is about a "download&upload" flow, and the direct
 Host <-> Host exchange will be another discussion related to the Sync
 Protocol.

 Now I am not so sure. Perhaps the architecture should already prioritize a
 conversational and asynchronous data exchange between sites, with ZIP
 downloads as a cool-but-secondary feature.


 > Lately I've been thinking that we could augment WXR since it's
 inherently expandable, to communicate much of the extra site meta. In
 fact, existing imports would/should ignore those sections, but transfers
 could use the information to aid in the import.

 I can see three "natural" ways of encoding that metadata:

 * WXR. However, when would WXR hold both content AND metadata? On site
 export the content would be in the database so the WXR file would only
 carry the metadata – at which point it's wouldn't have almost anything in
 common with WXR as we know it. On the upside, WXR can be streamed with the
 upcoming XML API and you can also edit them with a text editor.
 * JSON. Either package.json, wp.json, or blueprint.json. I am not sure
 what would it carry besides that metadata, though. A wp.json or
 blueprint.json file could be the standard bootstrap format for WordPress
 sites and it would play nicely with Blueprints being encoded as JSON.
 * SQLite. There could be a table like `wp_export_metadata` where all the
 relevant information would be stored. Or perhaps we could even store that
 as site options. It would be readable and writable via $wpdb.

 For ZIP exports, I would love the JSON option as it would be aligned with
 Blueprints JSON files. For streaming data exchange, WXR seems better
 suited. I'd scope this down to a single data format for starters and WXR –
 or XML in general – seems to have the most upsides. Later on we can chat
 about JSON support to repurpose the same logic in development
 environments.

 > One challenge in recording source and destination is that either can
 disappear.

 In ZIP exports, this is much easier as the archive contains all the static
 assets and not just their remote URIs.

 In direct Host <-> Host transfer we need an entire world of error handling
 logic.

 Now I question that fundamental question again. An initial implementation
 that only works with a ZIP export file seems like a much more attainable
 target. It would already help developers, small-to-medium sites, and
 Playground while enabling learning and exploring. The Host<->Host
 conversation could still drive all the major architectural decisions to
 enable smooth transition into implementing that API in V2.


 > Consider if we normalize to a Playground environment where every custom
 setting is adjusted

 I love this. My only concern are large, 10GB exports that wouldn't fit
 into memory today. Playground could support remote ZIP files as virtual
 filesystems, though, which would unlock working even with those huge
 sites. Writing the changes back into these ZIPs would mean simply
 appending updated files at the end of the stream.

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/60375#comment:8>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform