RightScale Blog

AWS is truly amazing.

I'm currently developing an open-source framework that allows theoretically infinite scalability utilizing round-robin dns, S3, EC2, S3DFS (allows mounting an S3 bucket as a local file system on multiple EC2 instances), SQLite (serverless embeddable database where each db is just a flat file).

S3DFS works at the block level and has read/write caching so is really fast. However, it requires a commercial license for non-personal use.

SQLite is amazing in it's power and simplicity. It will start to have issues on a high-traffic website, but I'm breaking each user into their own db file. That sounds really bad - but SQLite has a great feature that allows you to attach multiple databases together and run queries across them as if it's just one database (handy for site-wide stats, search indexing, etc). Also, since the db's are just flat-files - backing up is super-easy with tar gzip!

I'm using PHP for the coding, but anticipate other languages libraries to be built to use the system as well.

Help is welcome, just let me know if you're interested!

I was thinking of calling it infinizon - but that sounds kinda dorky - thoughts?

http://www.sqlite.org/

http://www.openfount.com/blog/s3dfs-for-ec2

Update: I just came across http://rightscale.com which is an amazing AWS console that allows you to control almost all aspects of EC2, S3, and SQS. Too many features to list here, but definitely a must see. I've been playing around with it and it works great.

What I like most about developing on EC2 is the ability to save an development environment as an Amazon Machine Image (AMI), and then spin up instances of it as needed. I can fine-tune the environment over time (just spin up a new instance, make changes, and save a new AMI), as well as have different environments for different types of development.

I think you could do something similar with Slicehost/Linode using disk images and configuration profiles, but it doesn't seem they offer anything like Amazon's machine images. Correct me if I'm wrong, though.

Of course, the flip-side of all this is that any individual instance of an AMI is inherently ephemeral, and you need to treat it as such. Even the reliability claims of EBS are fairly weak. [1] That means EC2 instance storage, even EBS storage attached to an instance, isn't truly persistent, and I haven't quite ironed out my system for dealing with this.

You'll probably want to run your own Git server [2] and commit code as it's completed. That still leaves the possibility of data loss if your instance dies after you've written code, but before you've made a commit. You could just take periodic snapshots of the EBS volume where your in-progress code resides, but whatever period you choose for the frequency (like every 30 minutes) is equal to the amount of work you could lose. And that sort of constant backup isn't really what snapshots are meant for anyway. [3] I think the ideal solution would be to run Dropbox to get…

Two reasons come to mind:

* the failure rate of an EBS volume (assuming a bug-free EBS software stack!) should be less than any one spinning disk

* these machines don't have disk space set aside for copying the root volumes to them from S3 to boot. (There's a 10GB limit, and to allow for this to be somewhere, you'd have to have another SSD, or partition the existing ones.)

There's a good blog post about it on Rightscale: http://blog.rightscale.com/2012/11/02/new-ec2-instance-types-and-coordinated-failures-in-the-cloud/

esh's "You Should Use EBS Boot Instances" also lays out a number of good reasons to use EBS for boot: http://alestic.com/2012/01/ec2-ebs-boot-recommended

You at least have two 120GB ephemeral SSDs to play with when you're running - a luxury you don't have on the new M3 classes.

"I/O requests cost $0.10 per million" :)
I wonder what is the amount MySql generates? Any data?

Edit, as I found this (http://blog.rightscale.com/2008/08/20/amazon-ebs-explained/):
"As a point of reference, our main database server is pretty busy and chugs along at an average of 17 transactions per second, which should total to around $4.40 per month. But our monitoring servers, prior to some recent optimizations, hammered the disks as fast as they would go at over 1000 random writes per second sustained 24×7. That would end up costing over $250 per month! As far as I can tell, for most situations the EBS transaction costs will be in the noise, but you can make it expensive if you’re not careful."

If you like this, you probably want to check out Rightscale's free package:

http://www.rightscale.com/

It's not as ajaxy, but RightScale's stuff can do a good bit more than Amazon's EC2 dashboard. Honestly, I was hoping Amazon would create something at least as good as free RightScale when I read this-- but it feels a bit worse than what most of us have been using for a good while.

This interface does look slick, though-- if they get it caught up in functionality I'd probably use it over RS. The release of this tool might reduce RightScale's valuation a tad since it does kind of indicate that Amazon's not in the market to buy someone like them.

Google has released "Committed Use Discounts", an RI competitor, which appears to be ~35% cheaper than RIs on average, according to Rightscale [0].

I also wrote an article discussing why "Committed Use Discounts" is a much bigger deal than just a "price model" at [1].

It'd be great for "thefht" guy to do the comparison, but data's already out there.

(work at Google Cloud)

[0] http://www.rightscale.com/blog/cloud-cost-analysis/aws-reserved-instances-vs-google-committed-use-discounts

[1] https://hackernoon.com/why-googles-answer-to-aws-reseved-instances-is-a-big-deal-4b9b36d8e631#.7p9z3grdh