[wp-hackers] Load Balancing | Media Uploads to Multiple Servers

[Eric Marden]

On Aug 5, 2009, at 1:29 PM, Steve Pellham wrote:

> Thanks for the input Eric...
>
> So what happens when the 'central location' gets cutoff from the  
> world:
> massive communication failure, hardware failure, fire, [insert your  
> own
> disaster here]?

You can always set up IP Failover of that box to another hot spare for  
that data. But there is quite a bit of difference between rsyncing  
stuff you don't have to, and having a central mount where changes are  
reflected live for all systems that read that mount. I've also had a  
lot of success using Fuse to mount S3. Performance is great, and you  
can configure it with a local cache in the case of S3 outage. And  
there is something really cool about having 200 TB+ mount point on  
your server. ;)

http://xentek.net/articles/448/installing-fuse-s3fs-and-sshfs-on-ubuntu/


> A central location for the site isn't acceptable if that 'central  
> location'
> (I'm guessing you mean some sort of NAS/Network Storage) gets cutoff  
> from
> the world.  I'm speaking in terms of separate www/db servers that are
> geographically dispersed.  If the 'web heads' both pulled from a  
> central
> location (easier to maintain... but...) there is still a single  
> point of
> failure.

Don't confuse High Availability with High Capacity with Load  
Balancing. They are distinct, and have different approaches to solve.


However, if you are trying to eliminate all points of failure, then  
you're going to need a few servers, with every service/server having a  
hot spare and redundant backups of data to be able to bring it back up  
quickly and reliably. Adding in geographically aware edge serving will  
complicate things, but your static content, including wp-content/ 
uploads, should probably live on a true CDN anyway if this is one of  
your concerns.

> I fail to see how not replicating data (mysql, application, content  
> data)
> across multiple servers prevents a single point of failure.

It seems that the more advice that comes in the more requirements get  
revealed. What I described can and will serve you and can be used in  
HC configurations, and includes a quick DR (Data Recovery) plan,  
provided you have off site backups of uploads (s3 is good for this)  
and could stand up another box in its place (or keep a hot spare  
handy) with minimal effort/downtime. Its all a balance between budget  
and practicality. In the scenario I've suggested and implemented  
dozens of times, its generally the web heads that get swamped and go  
down, not the DB/File server (which generally has more memory than the  
leaner web heads). MySQL replication is not as great as it could be  
and has its own trade offs that I like to avoid unless maximum  
failover is the name of the game.



- Eric Marden
__________________________________
http://xentek.net/code/wordpress/
http://hostedbyxentek.net