Anchor is an Australian sysadmin company whose blog covers Linux, storage, GitHub's architecture, rsync, and software failures.
HN commenters recommend it to sysadmins for good technical content and amusing criticism of software, and cite its posts on SSH ControlMaster and database issues.
This is also sometimes called 'checkpointing', and has been a long-studied feature necessary for distributed processing systems. There's a lot of ways to skin this cat. Kernel modules, application libraries, userland tools, and features to support various kinds of applications (single-threaded, multi-threaded, preserving TCP connections, etc). Find the one that fits your use best.
Descriptions of how checkpointing works: https://en.wikipedia.org/wiki/Application_checkpointing http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.117.4869&rep=rep1&type=pdf http://www.anchor.com.au/blog/2013/02/overview-of-checkpoint-and-restore-live-migrating-processes-on-a-linux-system/ https://lwn.net/Articles/478111/ https://www.usenix.org/legacy/event/usenix01/freenix01/full_papers/dieter/dieter_html/paper.html
Individual solutions: https://ckpt.wiki.kernel.org/index.php/Main_Page https://code.google.com/p/cryopid/ http://criu.org/Main_Page http://crd.lbl.gov/departments/computer-science/CLaSS/research/BLCR/
An almost-full list of solutions here: http://checkpointing.org/
> Why this isn’t the default I don’t know, [...]
I searched a little and found the following blog, which claims, that there are issues with it, when you do heavy data transfers, for example via many rsyncs at the same time:
https://www.anchor.com.au/blog/2010/02/ssh-controlmaster-the-good-the-bad-the-ugly/
I did not test it myself. This would seem like an appropriate reason to not make it the default.
Reminds me of the last Github post I'd read about their load-balancer setup from their server guy(2009):
http://www.anchor.com.au/blog/2009/10/load-balancing-at-github-why-ldirectord/
Willy Tarreau (author of HAproxy) had sparked a nice discussion in the comments section, that time.
If you're a heavy SSH user, using multiplexing in this manner can have negative consequences [1]. Downsides include having all your multiplexed connections exiting if the master exits!
[1] http://www.anchor.com.au/blog/2010/02/ssh-controlmaster-the-good-the-bad-the-ugly/
I was looking at http://www.anchor.com.au/blog/wp-content/uploads/2009/09/GitHubStorage_Small.png which shows file server B as a failover, traffic coming into the active LB.
Does the active LB then have a pool which includes other file server B services ?
Anchor's blog is worth subscribing to if you're a sysadmin. The technical content is good, and they can be merciless yet amusing when writing about software that fail to meet their standards of good taste (see http://www.anchor.com.au/blog/tag/fail/).