Michael Mitzenmacher

I certainly already agreed that Rails' architecture is bad (though the reason that it has this problem is its memory usage, and not any of the other reasons you mention). Herokus architecture is bad as well. It's the combination of these that causes the problem. But that does not mean that it's impossible, or even hard, to solve the problem at Herokus end.

> I'm not sure there are such providers, and if there aren't, I think it's safe to point the finger towards Rails.

This is not sound logic. I described above two methods for solving the problem: (1) increase the memory per Dyno (see below: they're doing this, going from 512MB to 1GB per Dyno IIRC, which although still low will be a great improvement if that means that your app can now run 2 concurrent processes per Dyno instead of 1), or (2) do intelligent routing for small groups of Dynos. Do you understand the problem with random routing, and why either of these two would solve it? If not you might find the paper I linked to previously very interesting:

"To motivate this survey, we begin with a simple problem that demonstrates a powerful fundamental idea. Suppose that n balls are thrown into n bins, with each ball choosing a bin independently and uniformly at random. Then the maximum load, or the largest number of balls in any bin, is approximately log n / log log n with high probability. Now suppose instead that the balls are placed sequentially, and each ball is placed in the least loaded of d >= 2 bins chosen…

> few applications of random walks for random human walking

I've heard a hypothesis that an advantage of dubious fortune-telling techniques (entrails, tea leaves, etc.) for hunter-gatherer societies is that random choice of prey and direction each day keeps them from over-exploiting any given resource to extinction.

A weird flex Indian Rajas were said to have done is to release a horse and then follow it, offering to fight anyone they met upon the resulting (not so random) walk. https://en.wikipedia.org/wiki/Ashvamedha

I have read both ancient sword fighters and modern fencers advise to consciously plan an attack, but unconsciously trigger it: by letting noise from the environment decide the moment T=0, you're less likely to give your opponent any tells.

One might imagine that divination before battle would provide what the modern world calls a "mixed strategy" and could indeed by superior to pure strategies. (compare the SSBN; see also https://www.atlanticcouncil.org/in-depth-research-reports/issue-brief/us-strategy-and-force-posture-for-an-era-of-nuclear-tripolarity/ )

On a more peaceful note, if you wish to choose between popular dinner spots, it may be best to make the selection at hazard: http://www.eecs.harvard.edu/~michaelm/postscripts/handbook2001.pdf

>assuming you have a perfect cryptographic hash function

Except there is no such thing (not in the sense you need it) - it too would be a perfect source of randomness, and you're just pushing the problem down the road.

As an example, NIST publication 800-90B recommends multiple randomness extractors, one of which is SHA. They recommend using twice the amount of entropy in as the entropy out to get random "enough" bits. Thus even SHA is not going to mix bits well enough as you want.

(The things called perfect hash functions in the literature map N items into N slots with no collisions, which is not what is needed here).

As a perhaps surprising counterexample to any simple solution, here [1] is a math paper with a proof that there can be no optimal algorithm that is best for all values of coin bias.

Here's [2] a cool walkthough on some of the ideas in theory that have been investigated.

[1] https://web.eecs.umich.edu/~qstout/abs/AnnProb84.html
[2] http://www.eecs.harvard.edu/~michaelm/coinflipext.pdf

Two random choices has a few significant advantages over "find the least filled one":

- It's O(1), rather than O(buckets).

- It's robust to stale data and concurrency. One way systems try to avoid the O(buckets) work is by caching, or by scanning in parallel, either of which cause stale load estimates. "Pick the best" tends to lead to significant over-filling http://www.eecs.harvard.edu/~michaelm/postscripts/handbook2001.pdf or https://brooker.co.za/blog/2012/01/17/two-random.html).

- Best-of-k (rather than best-of-two) provides a parameter `k` that allows the system to tradeoff between optimality (higher k is more optimal), and robustness to stale data and concurrent accesses (lower k is more robust). Mitzenmacher's core observation is that the step up from k=1 to k=2 is a massive improvement in optimality with only a small loss in robustness (and much more than most people would expect).

Good talk, I feel like I've tread exactly this same thinking and given a similar talk internally (i accidentally ended up as the LB SME), even used poisson distribution to model the requests and do some code demos.

He references a simple algorithm/paper on simply chosing between two servers, and the results on the smoothing out of variance was impressive (paper: https://www.eecs.harvard.edu/~michaelm/postscripts/mythesis.pdf ).

So, no real conclusion, it's just interesting to peel back the layers of LB.. so many people treat it as magic, when really its a statistical lie - and impossible to achieve perfect distribution (hence the title)!

It depends a bit on exactly what you mean.

But eg if you want to write a new hash table (with a new hash function), you'd want to do pretty similar-ish analysis to figure out whether it's a good idea.

I used some neat math in my time to justify much simpler algorithms than what we were using before. (But not hash table related.)

https://www.keithschwarz.com/darts-dice-coins/ is also a joy to read. (As mentioned in the discussion on https://news.ycombinator.com/item?id=8006336 )

Have a look at eg https://www.eecs.harvard.edu/~michaelm/postscripts/handbook2001.pdf for some more in this flavour.