CloudHarmony publishes benchmarks and status data comparing cloud servers, CDNs, and DNS providers.
HN commenters point to it as a tool for running benchmarks and for CDN performance comparisons, though one notes the site was having issues.
<disclaimer>I'm from Dyn.</disclaimer>
We're thrilled to see a new DNS performance monitoring tool in the public domain. It takes courage to release things like this to the world and open them up for critique, I applaud jimaek, the author, for doing just that.
As many have noted, the results are highly varied from many of the other reports we see today but more locations and tests are going to continue to improve accuracy.
The testing is synthetic, that is, it's run from within datacenters. I'd encourage anyone who can help provide/sponsor a VPS or resources to also do so. Providing consistent benchmarks against Authoritive Queries from know locations is great but the test network itself needs to expand to get truly meaningful comparisons between providers.
I'd hope to see real-user data also get added in the future, that is, testing on the end of home cable/broadband connections, these tests perform differently to synthetic tests but represent end-user experience more accurately.
It would also be great to see raw lookup performance between providers, exclusive of network latency.
For those interested in diving deeper into DNS performance, I'd also recommend reviewing Cloud Harmony's reports, the latest of which can be found here: https://cloudharmony.com/reports/state-of-the-cloud-dns-0315-report
For IAAS providers, you should care about the uptime of the regions (datacenter) that you are planning to use and the uptime of the specific services that your vms will use within those datacenters. In particular, I recommend paying close attention to the monthly/quarterly uptime trend in addition to aggregate uptime numbers.
And if you really care, try and find out the datacenter(s) your providers will be using and find out the historic performance of their different infrastructure and redundancy providers, and how often they actually test and upgrade their systems.
Look at the difference now when I filter by region and add the region data to the chart... it clearly shows just how hard these comparisons can be:
Provider Uptime Outages Regions Total Downtime
---------------------- -------- ------- -------- --------------
Google Compute Engine 100% 0 1 0 hours
Amazon EC2 99.9984% 7 3 0.13 hours
Microsoft Azure VMs 99.942% 56 6 2.89 hours
https://cloudharmony.com/status-1year-of-compute-in-america_north-group-by-regions-and-provider
Welp, there's a lot that needs addressing in this comment, but I'll stick to the heavy-hitters.
> they've had near daily network interruptions due to DDoS attacks since Christmas of last year, with some outages lasting up to almost a day
Whoa, nope[1].
> It also leaves them highly vulnerable to DDoS attacks through exchanges.
We aggressively de-peer with networks that regularly originate attack traffic, allowing us to size our ports according to utilization rather than worst case attack sizes. Multilateral peering is actually a bit more expensive than transit these days - much more expensive if you intend to significantly overprovision capacity.
> Who else can you name that utilizes up to 7 transit providers in a given city
That's fair, but missing context. We're figuring out who works best for our traffic profile. We will scale back/remove the underperformers and scale up those that prove their worth.
> It's not cost effective, is sub-optimal for resiliency against attacks, and fails to leverage peering effectively.
All of this is dramatically incorrect.
[1] https://cloudharmony.com/status
S3 SLA is actually for only three nines (99.9%) or 8.76 hours / year or 43.8 minutes / month of downtime: https://aws.amazon.com/s3/sla/
CloudFront offers the same availability. Many CDNs offer no more than three nines. Some claim 100%, but there will eventually be faults. Most do really well to not have recognized outages, but I nonetheless think they offer 100% to guarantee so that you always get credit for any downtime rather than guaranteeing they are never down.
You can look at replicating files to multiple providers; the following shows what kind of uptimes you can expect from the big players: https://cloudharmony.com/status-1year-of-storage
If you can live with read-only states with CDN; a similar report: https://cloudharmony.com/status-1year-of-cdn
Bigger, faster, more open, more secure ... what's missing from this list?
Oh, that's right, how about more uptime?
Provider Outages Total Downtime
---------------------- -------- --------------
Amazon EC2 12 2.01 hours
Google Compute Engine 65 3.27 hours
Microsoft Azure VMs 102 42.80 hours
Source: https://cloudharmony.com/status-1year-of-compute-group-by-regions-and-provider
And this doesn't show the other Microsoft cloud services that have even worse reliability.
At my company we invested a lot of time trying to make Azure Service Bus and Visual Studio Online work. Both were so dreadfully unreliable as to be unusable.
Namecheap is a fabulous hosting provider. Love 'em. That said, highly-redundant DDOS-resistant DNS infrastructure is not their focus--they offer it as a free service with hosting, but it's best to cough up a few extra bucks and move your DNS to something more robust with AnyCast (DNS Made Easy, Route53...) when you can if uptime's important to you.
Check this out for real-time query speed testing: http://cloudharmony.com/dnstest
(And yeah, looks like GoDaddy is a solid "Test failed" still...)
Funny note after running the test a few times--the worst performing provider is the one whose salespeople contact me the most...