Duckbill Group

It is crystal clear that this comment validates the misunderstanding of how bandwidth is priced in the real world and how, just because Amazon's price model has been reliable, that it must be a viable cost because people pay it. It's unfortunate that the reality you're basing this comment on is so far off base that you've convinced yourself AWS egress fees are somehow "stable". It's also very interesting that you're stating Cloudflare is under some unforeseen "pressure" because they're not turning a profit today. I would gather you're assuming Cloudflare is being soaked by infrastructure costs, which is completely incorrect, as they build out their pivot towards targeting the enterprise market by investing in their field (sales) and continually reinvesting in R&D.

It's also fantastic to read the misconceptions of the "value" AWS is providing. In some cases they do provide a much greater value over cost ratio, but if you've convinced yourself that blindly for AWS proper as a whole - boy do I feel for you the day you realize the economic advantage they manipulate to monopolize the cloud market, and not for the greater good of their customers.

Clear pricing you say? I'll use Corey Quinn (The Duckbill Group [0]) as an example again - his entire liveihood and business runs on the fact that AWS pricing is not even remotely clear. It's laughable that anyone would publicly make that statement at this point in time knowing what we know. Sure, if you're running a static site on…

Yeah, it's really just a way to talk through cloud architectural choices when you're designing for cost: committing to reserved instances for VMs (goodbye, eternal soul -- for at least the next three years), minimizing your data retention or migrating through storage classes for bucket data, and ruthlessly optimizing your use of per-usage services (bucket ops, API calls, serverless functions, and charge-per-op databases like Firebase, to name a few). Bandwidth and network services, however, are insanely expensive for what you get, show up all over your service architecture, and are hard to forecast, so they like to jump out of dark corners at the end of your billing cycle and mug you. (The ever-invaluable Duckbill does a nice job of laying out all the places where AWS levies its bandwidth tax: https://www.duckbillgroup.com/wp-content/uploads/2019/12/dbg010-datatxfr-infographic-20191219.jpg).

Depending on what your application is, this may never be an issue, or, if you're pushing heavy file and content traffic, it could form a huge chunk of variable costs that you need to figure out cost recovery for in your business model.

(All this gets upended if you're on something like SFDC or Dynamics, though, where costs looks nice and forecastable until you get to disk storage, where their pricing models appear to be put together by the kind of guys who usually present their invoices alongside a menacingly-brandished lead pipe.)

I would put the estimate of "server lying on the ground" at more like tens of billions of dollars.

Here's how I back in my math:

1. AWS is a $44 billion run-rate business.
2. This estimate from the Duckbill group has EC2 at 60% of the AWS bills of "large accounts". https://www.duckbillgroup.com/products/public-cloud-analysis/
3. I used to work in EC2 Networking and looked at the graphs with the server counts for all of EC2 on a weekly basis, so I can't say the actual numbers here, but I have a very good idea how big the EC2 fleet is.
4. 60% of 44 billion gives us an estimate of 26 billion run rate for just EC2.
5. In my time at Amazon where I was on multiple teams that ran large fleets (thousands of machines), and I heard about utilization numbers for services like Lambda and Fargate, 50% utilization would be a very, very good number for VMs in the cloud.
6. 50% of 26 billion is over 10 billion per year of wasted server capacity (that people are paying AWS for)

So yeah, tens of billions of dollars of AWS and the other cloud providers revenue are likely "waste" from the POV of customers.

Duckbill engineering is hiring a full-time ONSITE senior data engineer in downtown San Francisco. We WFH Tuesdays and Thursdays.

We're tackling the hardest finops challenges for the biggest enterprise customers head-on and have a killer team along with the advantage of the established and industry-leading DuckBill Group.

Help shape the leading edge of an entire industry with us while we're still just at seed stage and under 6 engineers!

Comp range is generous on the equity and between 180k and 210k base depending on seniority. We're cooking with flask, react, postgres, clickhouse, parquet and airflow on AWS at the moment.

---

Apply here: https://www.duckbillgroup.com/careers/

Or email: careers@duckbillhq.com