Netflix engineers write about the company's infrastructure, including containers, content delivery, data caching, notebooks, and security.
HN links it for Netflix engineering practices, such as Open Connect appliances, serverless-like platforms, and notebooks, and for its AWS credential security article.
Super cool! It reminds be of Netflix's Flamescope[0] also from Brendan Greggs work at https://medium.com/netflix-techblog/netflix-flamescope-a57ca19d47bb.
The reasons heatmaps are "news" is because not everyone has any formal exposure to data/stats curricula. The concept is easy, straight forward, and nearly obvious but unless you're in the position to have datasets to try to understand (such as a class where they have datasets and questions posed for you!) you just don't hear about this stuff.
What often happens is that you have a problem you're trying to understand: e.g. "Why does this crash?" or "Why is this slow on Wednesday?", and you use the data you have available to solve it. You don't normally have access to the kinds of data at the resolutions you need for something like heat-=maps to come into play. You're ops not a the application engineer and you just have 50-90-95-99 latency percentile graphs pre-aggregated in minute windows in nagios or graphana and maybe a few more of those for IO, NET, CPU, and thread counts and you're trying to correlate between these to form a hypothesis.
If it's important enough AND you have bad luck AND it can't be solved with more hardware, THEN you go deeper. You get to start actually trying to decide what new data is worth collecting ad-hoc, during what time intervals, etc. Only then do you even have data with the resolution where it's worth talking about heatmaps.
That's why this is and interesting reintroduction or new to…
Here is a blog post from Netflix staff members where they explain the recommendations: https://medium.com/netflix-techblog/netflix-recommendations-beyond-the-5-stars-part-1-55838468f429 . It's from 2012 so things might very well have changed since then.
apenwarr talks about this on his blog. He claims Netflix purposely switched from a good recommendation algorithm to a worse one after the transition from their DVD-by-mail days to their streaming offering. See https://apenwarr.ca/log/20190201 (find the section titled "Netflix").
I recommend reading that section (or even the whole article, though the main point of the article is something only tangentially related), but I'll try to summarize.
The argument goes that Netflix doesn't need to find the best movie or TV show for you; instead it tries to find one that is just good enough for you, but at the same time good enough for lots of other people too (apparently in your case they fail at that). From their standpoint it doesn't matter if you watch something you really really like instead of something mildly entertaining: Netflix noted that the probability of people cancelling depends on the number of hours they watch Netflix content, not on the quality of that content.
Nudging people to select what they watch out of a smaller pool of offerings enables them to save on bandwidth costs because a lot of content can be served out of distributed caches.
There's a word for it even, apparently: "satisficing"…
That really depends on the application, who has access to the keys, how they learn about the keys etc. There isn't a one size fits all solution and that's kinda why eventually consistency is a pain in the ass.
The "I want strong consistency but have to use S3 because its the cheapest way to store stuff and its great at serving reads and just otherwise great" option is to maintain an external index that maps between persistent ids (e.g. file paths) and S3 keys. We use this for hosting user content at my company. Our index is in MS SQL of all things ;)
The MVP option is to not to delete stuff. :) This can work well in some scenarios (e.g. you want to keep all versions around anyway) but be careful to not dig a hole that's hard to get out of.
A more esoteric option is lifecycle based cleanup. This can work well for things like a build artifact cache (where an expired key isn't a big deal because you can regenerate it.)
Some of this is discussed in this blog post by Netflix: https://medium.com/netflix-techblog/s3mper-consistency-in-the-cloud-b6a1076aa4f8 (caveat: s3mper is dead(?) and there is an AWS-native solution for HDFS now.)
The "pressure stall information" looks interesting. It's exponentially weighted moving averages of the fraction of time "some" or "full" (all non-idle) tasks are stalled on each of CPU, memory, and IO. Also a counter of absolute time. [1]
Printing this seems like a good addition to "Linux Performance Analysis in 60,000 Milliseconds". [2] More useful that the load average printed by uptime, which hasn't aged well. Linux mixes cpu-bound and io-bound stuff into one bucket [3] which is a terrible idea, and the number of tasks is confusing at best when you have a mix of thread-per-CPU and thread-per-connection stuff running.
Looks like it's also exported on a per-cgroups basis, which is great.
[1] http://git.cmpxchg.org/cgit.cgi/linux-psi.git/tree/Documentation/accounting/psi.txt
[2] https://medium.com/netflix-techblog/linux-performance-analysis-in-60-000-milliseconds-accc10403c55
[3] http://www.brendangregg.com/blog/2017-08-08/linux-load-averages.html
People might find this interesting to compare and constrast to (certainly I noticed parallels with[1]) Netflix's 'serverless' platform, discussed yesterday:
https://medium.com/netflix-techblog/developer-experience-lessons-operating-a-serverless-like-platform-at-netflix-part-ii-63a376c28228
Edit: As a sibling commenter notes this is most eminently doable with e.g. Common Lisp and BEAM (Erlang/Elixir), but more folks are (publicly) attempting this in other environments now (I've experimented with a number of approaches to this the last few years, so I'm trying to keep score - would love to see any comments on other attempts below).
[1]: Quote: "At the core of the redesign is a Dynamic Scripting Platform which provides us the ability to inject code into a running Java application at any time. This means we can alter the behavior of the application without a full scale deployment."
I don't think time efficiency is the primary concern for the big encoders like Netflix and YouTube. They're more interested in lowering the bitrate while maintaining image quality and are willing to throw compute resources at slower encoders which achieve that. Netflix, for example, does multiple encodes per scene in multiple formats to get the best possible quality for each format for each scene.
VP9 outperforms H.264 (libvpx versus x264) for Netflix's use case. Here are some articles from the Netflix TechBlog on their experiences with VP9 and their encoding approach in general:
https://medium.com/netflix-techblog/optimized-shot-based-encodes-now-streaming-4b9464204830
https://medium.com/netflix-techblog/more-efficient-mobile-encodes-for-netflix-downloads-625d7b082909
https://medium.com/netflix-techblog/netflix-downloads-on-android-d79db40f1732