Nick Craver

I'm not sure how many full handshakes/sec we're running - the logging isn't as readily available there because we're explicitly not logging the listeners involved on separate proc binds. We are logging the traffic on the port 80 front end they feed. We have tuned the performance there via best practices and taking into account very recent HAProxy changes for various SSL windows added by Willy, Emeric and help from Ilya Grigorik. They're all awesome and tremendously helpful. We are on a version slightly newer than 1.5.2 (we build straight from the repo).

Even with all that SSL we're only running around 15% CPU at peak so it's not having any trouble. Most of that CPU usage does come from the SSL termination though - it ran around 3-5% CPU before. We're also working on much larger infrastructure changes that mean SSL termination much more local to our users, which means the HAProxy load will drop to just about nothing again. I'm working on a followup to my previous SSL post: http://nickcraver.com/blog/2013/04/23/stackoverflow-com-the-road-to-ssl/ that will explain why it's not already on for all hits...I think hacker news will have fun with hat one.

All that being said, there's no reason we can't forward that syslog traffic from the listeners to our logging infrastructure to get at least a counter out of it. If you're curious how we log, I'll explain a bit.

We feed it into a logging display called Realog built by Kyle Brandt and Matt Jibson in Go. Here are a few snapshots of…

See https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-2016-edition/ for an overview of their stack.

They do NOT need a stack of daily traffic analytics hardware. As I said above, remote RPCs take an order of magnitude more resources than function calls. If you are used to using various distributed tools to do analytics, you are used to requiring an order of magnitude more hardware than a simpler solution. And if there are inefficiencies in your solution, it is easy for a second order of magnitude to sneak in. Therefore your experience is going to seriously mislead you about the likely limits of a simpler solution.

Making this concrete, if your job requires a Hadoop cluster of less than 50 machines, odds are good that it could be run just fine on a single machine with a different technology choice. But if you are fully utilizing hundreds of machines in your cluster, then you actually need to be distributed. (Though you could achieve considerable savings for some of your jobs could be run in a more efficient architecture. Amazon in particular makes excellent use of this kind of hybrid architecture in places.)

Google is making excellent decisions for the scale that they operate on. You just are unlikely to operate at a scale that makes their decisions make much sense for you.

Before anyone wants to criticize Stack Overflow, consider their server footprints as of 2016 [1]:

4 Microsoft SQL Servers (new hardware for 2 of them)
11 IIS Web Servers (new hardware)
2 Redis Servers (new hardware)
3 Tag Engine servers (new hardware for 2 of the 3)
3 Elasticsearch servers (same)
4 HAProxy Load Balancers (added 2 to support CloudFlare)
2 Networks (each a Nexus 5596 Core + 2232TM Fabric Extenders, upgraded to 10Gbps everywhere)
2 Fortinet 800C Firewalls (replaced Cisco 5525-X ASAs)
2 Cisco ASR-1001 Routers (replaced Cisco 3945 Routers)
2 Cisco ASR-1001-x Routers (new!)

Assuming they aren't lying, this is the entire list of machines needed for a site used by the world's developers. That's no small feat. Many companies whose developers blog about their "best practices" would struggle to achieve 1/10th as much efficiency.

[1] https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-2016-edition/

>You're looking at simple, static pages.

Stackoverflow pages have many dynamic components like vote counts, reputation points, sidebar related-questions-links, new comments, etc. An excerpt from their blog explains they can't cache the output :
https://nickcraver.com/blog/2019/08/06/stack-overflow-how-we-do-app-caching/#doing-more-harm-than-good

Even though Stackoverflow's website io access pattern has higher reads than writes, the resultant generated html is still not as static as Wikipedia pages. Even if they're efficiently using cpu to generate the dynamic elements, the cost of egress traffic may also be a factor.

All that said, I don't have any insight into what heuristics they use to block certain ip addresses.

Maybe just a couple. If you google the StackExchange Hardware Setup, you will find they use surprisingly little Hardware. Back in 2013, they could go with 2 single webservers not accounting for failover reduncancy [0]. They have upgraded since, but still, if you consider the amount of hits they get, they run a pretty humble hardware [1].

One of the arguments I make when people say microservices + cloud built to the core is the only way to scale - clearly a cleverly architected approach can save you lots on hardware/hosting money.

[0]: https://nickcraver.com/blog/2013/11/22/what-it-takes-to-run-stack-overflow/#core-hardware
[1]: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-2016-edition/

OIDC integration, RBAC, data persistence/replication across nodes, observability, mTLS.

And yes, you're right, all these things are complex in any situation, except when you simply use a load balancer and two servers. There are companies estimated to be worth close to US$2B using with sub 30 servers using a service you may have heard of: Stack Overflow (https://nickcraver.com/blog/2016/03/29/stack-overflow-the-hardware-2016-edition/ && https://nickcraver.com/blog/2016/05/03/stack-overflow-how-we-do-deployment-2016-edition/).

(As an aside: K8s does not free you from your Cloud vendor - after floating around my city working on various "platforms", all locked in due to IAM, for example.)