[wp-trac] [WordPress Trac] #22981: Tweets import plugin tracking ticket
WordPress Trac
noreply at wordpress.org
Sun Feb 2 09:28:39 UTC 2014
#22981: Tweets import plugin tracking ticket
----------------------------+----------------------------
Reporter: nacin | Owner: PeteMall
Type: task (blessed) | Status: accepted
Priority: high | Milestone: WordPress.org
Component: Import | Version:
Severity: normal | Resolution:
Keywords: | Focuses:
----------------------------+----------------------------
Comment (by Otto42):
wp-twitter-importer.3.php is my modification of the last version uploaded
by aaroncampbell.
- Removed the dependency on the WP_Importer_Cron (for now, the plugin
wasn't really using it anyway). May add this back later, along with
checking for duplicates on import and smarter parsing.
- Some start and end functions to speed up the process by turning off the
cache invalidation and term/comment counting during the import
- Speed boost by turning off autocommit and manually committing once every
1000 tweets. This isn't the safest thing in the world, but it's a start.
The speed boost is substantial. Importing 11,000 tweets went down from ~6
minutes to ~30 seconds.
- Added a filter for the post_type used, in case somebody doesn't want
them all to be "post".
Few known problems:
- It's using the WP_Filesystem slightly incorrectly. I'm not sure how to
fix this at present because I've never tried to use it with a file upload.
Think the order of operations needs to be changed around here. It will
work with "direct" mode only at present.
- It's still a bit slow and will run into PHP timeouts. There are some
enhancements that can be made for speed, and it can be put into a
continuous cron job.
- Turning off autocommit with 'set autocommit = 0' strikes me as
potentially bad. Probably need to find a better way.
- Like the other importers, we should probably make a token effort of some
kind to download the media "attachments" in the tweets and rejigger the
URLs to local ones. Might want to limit this to known domains, like
pic.twitter and the like.
--
Ticket URL: <https://core.trac.wordpress.org/ticket/22981#comment:38>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform
More information about the wp-trac
mailing list