Varnish Software's company blog covering the Varnish Cache HTTP accelerator, its features, and related tools like TinyKVM and Zipnish.
HN commenters like Varnish but complain that the best features go only into the pricey commercial version, and note past gaps such as late HTTP/2 support.
Excellent tour through this topic, here's my best attempt at a tl;dr summary:
1. write a 'Hello World' HTTP server to show how Tokio's async-Rust primitives work for epoll-based I/O;
2. show how the tracing crate ecosystem works to inspect and debug async-runtime tasks;
3. play with converting traces to OpenTelemetry, tokio-console, etc. for different views of the collected data;
4. play with hyper and axum to build a simple HTTP service that makes an upstream HTTP request to the YouTube API;
5. cache results by ID so it can shared across all request-handlers;
6. add 'request coalescing' to prevent multiple concurrent requests from making upstream requests in parallel;
7. fix a bug in the request-coalescing implementation that caused coalesced requests to hang when the in-flight request raised an error.
One word of caution about using any request-coalescing pattern like this on a highly-loaded server: in the case where every request fails to populate the cache (e.g., timeout / not found), you end up in a situation where all requests end up _serialized_ and can quickly lock up a server under constant load. You need to either add a circuit-breaker, or otherwise ensure that _something_ always gets cached in any otherwise-uncacheable edge case (e.g., hit-for-pass in Varnish [1].)
[1] https://info.varnish-software.com/blog/hit-for-pass-varnish-cache
I'll echo what Simon said; we share some experiences here. There's a potential footgun, though, anyone getting started with this should know about-
Request coalescing can be incredibly beneficial for cacheable content, but for uncacheable content you need to turn it off! Otherwise you'll cause your cache server to serialize requests to your backend for it. Let's imagine a piece of uncacheable content takes one second for your backend to generate. What happens if your users request it at a rate of twice a second? Those requests are going to start piling up, breaking page loads for your users while your backend servers sit idle.
If you are using Varnish, the hit-for-miss concept addresses this. However, it's easy to implement wrong when you start writing your own VCL. Be sure to read https://info.varnish-software.com/blog/hit-for-miss-and-why-a-null-ttl-is-bad-for-you and related posts. My general answer to getting your VCL correct is writing tests, but this is a tricky behavior to validate.
I'm unsure how nginx's caching handles this, which would make me nervous using the proxy_cache_lock directive for locations with a mix of cacheable and uncacheable content.
I was spec-ing out Varnish for a project recently and was really impressed with what it could do. The stale-while-revalidate approach (immediately serve data past the `ttl` for a `grace` period while asynchronously refreshing it) keeps data flowing. It can `keep` data past the `grace` period and serve that when the origin goes down or use that in conditional requests (If-Modified-Since) to the origin to prevent fetching unchanged content. Coalescing requests mitigates thundering herds that reach stale caches. VCL scripting makes the whole thing very configurable. I think Varnish needs better marketing, but I suppose Fastly does that now.
[1] https://www.varnish-software.com/developers/tutorials/http-caching-basics/
[2] https://info.varnish-software.com/blog/request-coalescing-and-other-reasons-to-use-varnish-as-an-origin-shield
[3] https://docs.varnish-software.com/tutorials/object-lifetime/
[4] https://www.varnish-software.com/developers/tutorials/varnish-builtin-vcl/
That's really neat that you can do the authentication/authorization headers within Varnish.
I'd still use CloudFront instead, which gives you these features plus the advantage of having global edge servers, and support for TLS/SSL (not sure about the http/2 part).
Although the one advantage this has is the cache can stop you from having to leave your external network to return a file (vs going to CloudFront or S3). That can be useful.
Another commenter pointed out that Varnish won't support TLS either, which is correct. The funny part about that is while they have long had the "why not SSL?" FAQ page espousing the terribleness of OpenSSL, they DO offer that feature on their paid product Varnish Plus. That not being a basic feature is a tad crazy.
[1] Why no ssl? https://www.varnish-cache.org/docs/4.1/phk/ssl.html
[2] http://info.varnish-software.com/blog/how-implement-ssltls-varnish-plus
I borderline love Varnish cache, but the way the open source and commercial versions have diverged and all the nice features only go into the $10k/yr+ commercial version sucks.
Features like the memory governer [0], because trying to predict how much memory (open-source) Varnish will use is an absolute PITA and a sure-fire way to run out of memory if you're not careful.
My clients can't justify the commercial license costs (as a sibling comment says, CDNs eat Varnish's lunch in that market) and yet what I can do with Varnish and the power it gives me makes it real magic.
It would be nice to see a modern reimplementation of Varnish, open-sourced, but I doubt that would ever happen.
0. https://info.varnish-software.com/blog/two-minute-tech-tuesdays-memory-governor
Interesting; I'd love to read _what_ exactly they send in those long HTTP requests.
I'd think something along the lines of Surrogate Keys/Cache Tags would make sense:
- https://www.drupal.org/developing/api/8/cache/tags
- http://info.varnish-software.com/blog/advanced-cache-invalidation-strategies (ugh, they broke their site, cached at https://pinboard.in/cached/a31c4630dd2d/)
- https://www.fastly.com/blog/surrogate-keys-part-1
- https://blog.cloudflare.com/introducing-a-powerful-way-to-purge-cache-on-cloudflare-purge-by-cache-tag/