Aerospike, a NoSQL database company, writes about its key-value store, benchmarks, and performance comparisons with Redis and Cassandra.
HN links it when discussing low-latency distributed key-value stores. Commenters compare Aerospike to KeyDB and RethinkDB and note its use on raw SSDs.
Hi Gary4gar,
Read the points that you stated in your question. If you focus on growth (clearly your wins are going to bring more data updates as your increase the customer base) you should look into a solution that does not tie you down with data model and also does not expect you to worry about sharding in the future. Having done sharding and when one keeps growing the data volume and having to do shards it becomes painful and hurts the development flow. Been there many time and done it quite a bit myself. (Am a technical guys myself, http://www.linkedin.com/in/sukantag)
Look for a NoSQL solution which will help in the flexi-schema model. Helps you add different types of data models relatively easily. Check out Aerospike (http://www.aerospike.com), the product scales like scaling problems have disappeared from this planet. All of the characteristics that you would need based on what you have stated is in the product. Best of all it is free for startups. Just give it shot. Open Source server and client libraries in multiple programming languages. (Am a big "C" and Python guys so like them client libs a lot). Lots of startups and many large software companies use them.
If you like it and works for your application tell others, if not tell us what did not work.
I ain't sellin and you ain't buyin;
Ciao
You may also want to consider Aerospike database.
Aerospike is an open-source database known for its high performance. If you can represent your timestamp as an integer, you can built a secondary index on top and do a range query on it.
The range definition looks something like this:
Filter.range(name, begin, end) )
More details: http://www.aerospike.com/docs/client/java/usage/query/query.html
The advantages of Aerospike is its going to be superfast as the index is co-located with the data and also the secondary indexes are always updated inline with writes. So, the indexes always reflect the current state of truth. While doing the secondary index query, its follows a scatter gather approach thereby achieving distributed parallelism of the query. More details: http://www.aerospike.com/docs/architecture/secondary-index.html
On top of this, if at all you need to perform an aggregation on the secondary query result, you can use the Aerospike's Distributed Streaming Aggregation framework (http://www.aerospike.com/docs/guide/aggregation.html). Where you an do the aggregation on the server side itself without pulling all the data to the client. Only the final aggregation happen on the client.
Unlikely you are going to overwhelm Redis in a non-deterministic fashion. What do you even mean by overwhelm?
The post only concerns itself with in-memory workloads; I don't think Aerospike is competition in this space, while their advantage is against workloads working against SSD backed datasets. "After Google published a blog post “Cassandra Hits One Million Writes Per Second on Google Compute Engine” – using 300 nodes, we followed the same steps and documented how Aerospike Hits One Million Writes Per Second With Just 50 Nodes On Google Compute Engine. [...] Aerospike on SSD is very comparable to that of RAM. At 100% write, the SSDs are able to sustain 226,000 transactions per second compared to 239,000 for RAM." [0] Redis would have no problem hitting 250K IOPS on just a single core provided by GCE (or AWS EC2 or similar).
[0] http://www.aerospike.com/resource/aerospike-soars-in-the-google-cloud/
As a Cassandra Committer, I would really appreciate you updating your ycsb integration so we can corroborate your benchmarks, as I currently doubt their authenticity/honesty. It looks very much like you compared non-durable performance in Aerospike to durable performance in Cassandra.
It also appears from your documentation that you do not support any kind of safe active-active multi-dc mode, even in your paid-for offering (http://www.aerospike.com/docs/architecture/xdr.html), so even if faster than Cassandra, users should carefully read the fine print before deciding to use Aerospike, unless as the application grows in size and necessity (i.e. uninterruptibility in event of disaster) you want to find multi-dc is not actually viable for you.
NB: I'm unaware of any adform contributions to Cassandra, although a senior developer there has filed a few bug reports.
How would you model timeseries on top of Aerospike?
Using 1KB chunks of datapoints as simple K/V?
Or using Large Data Types [1] like Large Stack [2]?
Large Stack
A Large Stack collection is naturally aligned with time series data
because the stack preserves the insert order.
Stacks provide Last In First Out (LIFO) order,
so it's a convenient mechanism for tracking "recent" data.
Usage examples include:
Show me the last N events
Search the last N actions with a filter
Show me all documents in reverse insert order
[1] https://www.aerospike.com/docs/guide/ldt.html
[2] https://www.aerospike.com/docs/client/java/usage/ldt/ldt.html
Aerospike - distributed key value store has a little bit different of a use case than RethinkDB - distributed JSON document databases.
Basically the only valid use that I have seen for Aerospike, and that is the one that they are advertising is a distributed key value profile store for Ad-tech or marketing companies (http://www.aerospike.com/overview/).
Keep in mind that to actually use all of Aerospike's features , especially the one they are really proud about - the cross datacenter replication - you need the commercial license.
So..I suggest you figure out what your requirements are and then use the best tool for the job.