Peter Bailis

This is a bit dumbed down, and ignores the domain terminology required to properly discuss the trade-offs here (which is puzzling given that it links to a post by Aphyr, where you can find incredibly thorough discussions around isolation levels and anomalies).

> The fundamental problem with using Kafka as your primary data store is it provides no isolation.

This is false. I can only assume the author doesn't know about the Kafka transactions feature?

To be specific, Kafka's transaction machinery offers read-committed isolation, and you get read-uncommitted by default if you don't opt-in to use that transaction machinery (the docs: https://kafka.apache.org/0110/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html). Depending on your workload, read-committed might be sufficient for correctness, in which case you can absolutely use Kafka as your database.

Of course, proving that your application is sound with just read-committed isolation is can be challenging, not to mention testing that your application continues to be sound as new features are added.

Because of that, in general I think that the underlying point of this article is probably correct, in that you probably shouldn't use Kafka as your database -- but for certain applications / use-cases it's a completely valid system design choice.

More generally this is an area that many applications get wrong by using the wrong isolation levels, because most frameworks encourage incorrect implementations…

> Cassandra is eventually consistent. Consistency is pretty important.

Your above statement unfortunately has no match in distributed systems theory or practice. Basically eventually consistent means that the system is not consistent from the point of view of CAP, so it does not feature strong consistency.

As an alternative eventual consistency means: our contract with the user is, that, at least, when there are no partitions, the system will no show split-brain conditions, so a given information will converge to a given value for all the nodes.

This alone is a contract with the user, but does not tell a lot, it just tells you:

1) No strong consistency, sorry.
2) No split-brain conditions when the partitions heal.

So "It features eventual consistency, consistency is important" statement, I'm afraid, means that you don't have the tradeoffs about distributed systems clear.

Now, if EC alone does not mean a lot, how to classify an EC system? It depends on the exact implementation of what happens when there is a partition, and what happens when the partition heals.

There are EC systems that are configured by default to accept writes in the minority side (even in partitions when there is a single DB node if we want to honor "A" of CAP), and later will merge depending on "wall clock" timestamps.

This is a form of consistency (and is weak), but is not Consistency with capital-C of CAP.

Other systems will use vector clocks instead of timestamps, which is stronger but slower.…

I've been using Rails for a decade now, across many jobs, and one thing I've experienced fairly consistently is the worst Ruby web apps I have ever dealt with are the ones that weren't written in Rails.

I've both seen and spent a lot of time rewriting overgrown Sinatra monstrosities with assorted non-AR ORMs. These apps were generally riddled with XSS since they didn't handle escaping correctly, often had unmaintainable code that sometimes was thousands of lines long crammed into a single file, and lacking any sort of basic structuring principles mixed parameter handling with business logic all over the place.

I've seen several attempts to "build a better Rails" that's "more OO", but even frameworks that solve the problems I've just described in the previous paragraph have one huge drawback: Rails is a lingua franca for web development in Ruby, and whatever incremental gains you get from having a better framework are heavily outweighed by the costs of making people learn a new framework-per-project, the lack of features and documentation, and the lack of the library ecosystem that surrounds Rails.

At points I've tried to keep three ORMs/DB libraries in my head at a time: AR, DataMapper, and Sequel, which just left me wishing I was dealing with AR (the devil you know...)

ORMs need massive feature sets to jam the proverbial square peg of relational databases through the round hole of objects, and AR is the only one I've used that's both expressive enough to cover all the…

Looks like you cross-posted this comment here and on my blog, so I'll also cross-post in my response.

I am quite confused by your statement that YugaByte does not claim linearizability. The C of CAP is linearizability (see the original CAP theorem https://users.ece.cmu.edu/~adrian/731-sp04/readings/GL-cap.pdf). By claiming to be CP from CAP, you are claiming linearizability.

CockroachDB also makes the same CP claim. However, they explicitly walk back from this claim (https://www.cockroachlabs.com/blog/living-without-atomic-clocks/). In contrast, YugaByte documentation makes no effort to walk back from this claim. Rather, the documentation seems to indicate that YugaByte is linearizable at: https://blog.yugabyte.com/jepsen-testing-on-yugabyte-db-database/ and https://docs.yugabyte.com/latest/develop/learn/acid-transactions/.

Also, I would encourage you to read the Herlihy-Wing paper on linearizability published in 1990. You seem to be confusing linearizability with ACID isolation levels, which are actually a different concept. Peter Bailis has a good blog post on the difference: http://www.bailis.org/blog/linearizability-versus-serializability/.

On your third point, we agree. However, the point of my post is that relying on max clock skew without hardware support is dangerous. I’m going so far as to say that it’s so dangerous that it is incorrect to claim consistency guarantees when you make such assumptions.

I would really encourage YugaByte to consider using global…

It's not surprising that a simple theorem doesn't deal with practical trade-offs. In my experience if you feel you are running up against the CAP theorem you need to take a moment to think about the real needs of your application and looking for a more detailed theorem isn't going to guide you well, instead you need to think about what you actually want to happen when various kinds of failure occur. In finance for example, often trading systems will choose to be available and accept transactions when there is a failure and then reconcile afterwards - swapping the risk of making mistakes but in a trusted environment to prevent the potentially catastrophic condition of not being able to trade.

The article claims that the CAP theorem only prohibits linearisability as a consistency model. The original CAP paper talks about "Atomic consistency" which is clearly at least linearisability. However as the proof formalizes, it's obvious that if you accept writes to totally split and separated systems which cannot communicate together (network partition), you cannot be consistent across the two parts. This is confusing because this kind of consistency only loosely matches the definitions in the ACID acronym which are geared around invalid database states in a single machine database, not about different requests to different database machines giving inconsistent answers.

Linearizability versus serialisability (and strict serialisability) are explained well by Peter Bailis…

I don't see the connection between your reply and the linked article's content. A major point is already made right at the start when an ACID definition is quoted from a book - which makes it clear that that term is too nebulous and, in conclusion, that simply regurgitating some definition is not helpful in practice.

On the other hand, if you are actually able to design for all those complex points you may still not have one short definition to quote available, which would not speak against you at all. Not having a short definition for ACID to quote may happen to both real experts as well as people who really don't know anything. Because for real experts that definition is too simplified to be of any use.

I saw similar issues a lot in school and in academia, when having to answer some test questions: When you actually know a lot more you may actually get many answers wrong, because the one asking used a much simplified model (and therefore more inaccurate than yours) as basis for creating their question. For example in biology, there are some key tenets, but when you go deeper you learn about all the additional quirks and exceptions. Using the simplified tenet an answer to a question "can this or that happen" would be never, but for someone who knows a lot more that answer would be wrong. As an expert, you no longer use that simplified model you learned early in school.

Example blog post showing the world is more complex than the simple definition:…