Willy Tarreau

> If you start with GPL stuff (and in my experience most of the really good open source software is GPL)

Much as I prefer the GPL to BSD licences, that sounds odd to me, unless you mean for pretty small values of most. Some best-in-class Free software more or less off the top of my head, that isn't GPL:

apache (most everything, but lets stick with
httpd and solr)
nginx
openssh
freebsd
openbsd
openssl
ghc
go (golang)
plan9
PostgreSQL
redis
riak
elasticsearch
clang/llvm
v8/chrome

If you add inn LGPL the list grows, perhaps notably to include erlang and to a certain extent haproxy (I didn't realize licensing had changed, the rationale may be interesting in this context[1]).

I guess there are a couple of ways to productize GPL software -- one is the RedHat/Canonical approach: use (rather large) resources on testing and creating a coherent whole, including branding (arguably RedHat does more work here than Canonical -- but Canonical certainly aren't the first to try to value-add on top of Debian). The other way is to sell an appliance of some kind. You could say the kindle and even Android fall under this category (as does haproxy, I guess).

> In such a scenario, any business that needed an off-the-shelf 3D editor or image editor but also developed in-house software (for example, Pixar) would find it in their interest to patch bugs and generally improve the software that they use, even though they don't sell…

I'd suggest writing an add-on or utility for a popular data-managing server. You'll learn a lot and perhaps create something many others can use without having to get in too deep.

Here are some suggestions. I personally like the databases listed but I put the cache stuff first since thats what you seem to like.

Cache / Proxy tools:
http://www.squid-cache.org/
http://haproxy.1wt.eu/

Web Servers:
http://www.apache.org/
http://www.lighttpd.net/
http://nginx.net/
http://www.acme.com/software/thttpd/

Databases:
http://www.postgresql.org/
http://www.sqlite.org/

Version Control Systems:
http://subversion.tigris.org/
http://www.selenic.com/mercurial/wiki/
http://darcs.net/
http://git.or.cz/

Yeah, you want leastconn + app/router affinity. Affinity is the statement that all of your requests for an app go through one router (to avoid distributed connection state tracking).

In the past, I've accomplished this by having the next layer up consistent hash the api keys onto the router list. If you don't control the top layer (ELB), you need to add a dumb layer just for the hashing.

HAProxy works great for this extra layer. In practice, all you end up doing is adding a "balance hdr(host)" directive (see http://haproxy.1wt.eu/download/1.5/doc/configuration.txt) to get the hashing right, and you're spending <1ms inside HAProxy.

The Varnish authors have specifically stated they didn't want to introduce SSL into their process because of the added attack surface that adds. [0]

If you want SSL for Varnish, use something like Pound [1], stunnel [2], or HAProxy [3] to do the SSL termination and pass it off to Varnish...

I'm partial to Pound because it is lightweight and doesn't try to do any caching, it simply terminates the SSL and passes the request on.

[0] https://www.varnish-cache.org/docs/trunk/phk/ssl.html

[1] http://www.apsis.ch/pound

[2] https://www.stunnel.org/index.html

[3] http://haproxy.1wt.eu

I recommend starting from reading the HAProxy site.
http://haproxy.1wt.eu/

It has a ton of information about building a load balanced infrastructure using HAProxy.

This article helped me too: http://horms.net/projects/has/html/node6.html

Read some architecture articles from this site: http://highscalability.com

I am also planning on doing that, so once suppliers allow multiple machines I'll write a blog post about I had to do.