Netflix engineers write about cloud infrastructure, resilience testing, performance analysis, and recommendations, with early posts on Chaos Monkey and AWS.
HN commenters point to it as a source of more technical detail on Netflix and list it among worthwhile company engineering blogs.
That's a somewhat difficult questions to answer, and it depends on which aspect of EC2 you're interested in. EC2 isn't really a thing for which there are tutorials because EC2 is essentially a server to rent, It's how you use EC2, and there's hardly a standard for that currently. I think the more important thing is to become familiar with the Amazon Web Services (AWS) as a whole, of which EC2 is only a part. There are some AWS SDK's which I find wildly helpful. Here's the docs for the php sdk http://docs.amazonwebservices.com/AWSSDKforPHP/latest/#i=AmazonEC2
following other people companies who are using AWS can be a great way discover ways of using AWS. Netflix operates a great blog. http://techblog.netflix.com/
If you're interested in in just renting a server, and putting Wordpress on it, there are tutorials for that, and you'll have to deal with setting up a web server, domain name etc... But perhaps you're interested in learning about how build out a scalable protein folding cluster, and the set of challenges will be completely different. AWS only provides the hardware and you must create whatever you wish to implement.
Los Gatos, CA - Netflix
My team at Netflix is looking for Site Reliability Engineers to come in and help solve our availability and reliability challenges. We have one of the largest AWS footprints out there and are constantly finding out about the limits of AWS technologies.
We have a unique problem that we want to figure out how to enable all of our engineering teams to deploy faster and more often, while still increasing the overall availability of the Netflix streaming service. We do this by building tools/services for engineering teams and by helping teams figure out ways to make their software more reliable.
Official Job Description:
http://jobs.netflix.com/jobsListing.html?id=oHxbWfw5
To get a sense of what engineering teams at Netflix work on:
http://techblog.netflix.com/
https://github.com/netflix
If interested, please feel free to email me directly (email in profile)
During the evening hours, Netflix represents over 35% of all internet traffic in the US. That is an insane demand to be able to service, but they've done extremely well in my opinion. Much of their tech is open-sourced leading to much better practices industry-wide and they're transparent when things go wrong.
Here's an example of their preemptive work to deal with the loss of AWS/ELB regions:
http://techblog.netflix.com/2013/12/active-active-for-multi-regional.html
An an example of their post-mortem after deploying Node.js with a bad handler:
http://techblog.netflix.com/2014/11/nodejs-in-flames.html
Is there any other service with a library well over a petabyte in size that delivers multiple terabits/second worldwide to millions of customers with better reliability? Maybe YouTube with the backing of a company worth $300B more than Netflix?
In this case, I would still recommend Cassandra. It can easily handler the data sizes you mention as well as the write rates you imply further down the thread.
Cassandra has a nice and simple architecture (every node is identical, no zookeeper roles etc), high write performance and scalability [1], and is fairly robust. My main piece of advice is to get the tables correctly set up. You need to know exactly what queries you want to make and design a table around that query (Cassandra only allows performant queries to be made, unless you go out of your way to set a flag). Whether a query is possible or performant depends on the key of the rows for the table, which may be a composite key. Take a look at the cassandra documentation for more details.
1. http://techblog.netflix.com/2011/11/benchmarking-cassandra-scalability-on.html
I had asked a similar question on Quora [1] few years back. Here is the summary:
- Facebook: https://www.facebook.com/Engineering?sk=notes
- Google: http://google-engtools.blogspot.com/
- Twitter: http://engineering.twitter.com/
- Linkedin: http://engineering.linkedin.com/blog
- Dropbox: https://tech.dropbox.com/
- Instagram: http://instagram-engineering.tumblr.com/
- Netflix: http://techblog.netflix.com/
- Flickr: http://code.flickr.com
- Etsy: http://codeascraft.etsy.com/
- Yelp: http://engineeringblog.yelp.com/
- Heroku: https://blog.heroku.com/
- Airbnb: http://nerds.airbnb.com/
- Bitly: http://word.bitly.com/
- Evernote: https://blog.evernote.com/tech/
- Highscalability: http://highscalability.com/
[1] https://www.quora.com/What-are-the-companies-that-have-some-of-the-best-engineering-blogs
Is it just me or has the Netflix engineering team really been cranking out good stuff as of recent times? I mean holy crap. This thing feels so futuristic just watching it. I can't begin to express how blown away I am.
Given how crazy creative the netflix team is, it's not a stretch to imagine their next hackathon. Put this on a big screen and sit in front of it. Throw in a few hazard lights and a siren and music that plays automatically the moment there's a failure and it's a done deal.
Side note: If you want to get a better understanding of what you are reading, the inline linked article[1] is a must read
Also, I actually thought the pain suit was a real thing for a moment. Well played Netflix. Well played.
[1]http://techblog.netflix.com/2015/09/chaos-engineering-upgraded.html