Jepsen

If the recipient fails after telling the deduping machine what it has seen, the recipient's failover will be in an unknown different state. And now the failover machine has to try to figure out what its state should be, what is going on, and so on. You can add ways to addressing each possibility here, and you'll get ever more obscure chains that again can result in failure. At the cost of ever increasing complexity, you'll make failures ever more complicated.

Frequently developers who implement these manage to do two bad things. First they introduced a lot of complex code which rarely triggers except in disaster, and so whose bugs tend to survive. And second, they manage to convince themselves that they have accomplished the impossible, and make reliability promises that other developers unwisely believe. Exactly how unreliable most systems were and how much the documentation couldn't be trusted was underappreciated until https://jepsen.io/ came along and started proving how bad most distributed software was.

Now it may seem bizarrely unlikely that you'll ever see this kind of situation. But failures often start from network congestion due to a packet storm. And the failures lead to chatty Byzantine fault tolerance protocols adding significant traffic. This causes cascading failures. And so a small, simple outage can escalate into a series of outages as ever more confused servers continue overwhelming the network with their futile attempts to discover what is…

Thank you! And I agree haha. I love writing SQLite VFS's.

In SQLSync, the major issue was precisely what you describe: pulling in the full snapshot to replay. This is the main driver behind the "partial" aspect of the Graft design. It means that clients only need to pull the portion of the incoming snapshot that they don't already have and that overlap with the read/write set of their transactions. So yes, to answer your question once SQLSync is powered by Graft, it will frequently be able to avoid downloading all state in order to reset and replay.

Note that if a client is ok with relaxing to snapshot isolation (in particular this means clients may experience Write Skew[1]), and the r/w set of their local transaction does not intersect the snapshot changeset, Graft is able to perform an automatic merge.

[1]: https://jepsen.io/consistency/phenomena/a5b

Although I'm not a fauna user, I like studying databases and query languages. I've followed them since Jepsen¹ reviewed FaunaDB 2.5.4 back in 2019.

While their FQL looks useful, I can't imagine using a SaaS-only proprietary query language in this age for anything other than a research proof-of-concept to learn more about my problem before building a real product on an open platform.

Curious that Bob Muglia (ex Snowflake) is on their board. I believe he's also on the board of Relational AI which is another SaaS with an interesting but proprietary query language. Snowflake is a huge success in spite of being proprietary because its SQL isn't hard to learn or use, but I can't say the same about rAI. Makes me wonder how they will turn out.

¹ https://jepsen.io/analyses/faunadb-2.5.4

The price of persistent storage and memory was the same for Mongo as it was for Postgres back when the NoSQL movement happened. Mongo wasn't made of magic scaling fairies and still needed resources. As soon as you cranked up the safety of Mongo to a nearly acceptable level it's performance fell through the floor. The only reason people though it performed amazingly was because of Mongo's rather deceptive marketing.

I use the term "nearly acceptable" because for a long time Mongo's data consistency was absolutely crap (https://jepsen.io/analyses)

Personally I think people used Mongo because it was new, shiny and exciting and it promised performance (though it didn't really deliver for most people).

I disagree that the analysis articles from Jepsen, taken as whole, are arguing for a "perfect distributed system without trade offs". In my experience these articles are fairly even-handed, indeed, many products make some changes based on the analysis and then contract for a follow-up. That the "tone and implication is as if somehow there is some alternative panacea without tradeoffs..." is not something I observed in this analysis or the previous ones that I have read.

In my opinion the Rethink DB 2.1.5 report went fairly well for that project. If the claims are aligned with the reality of the product, it's clear the Jepsen report will highlight that.

https://jepsen.io/analyses/rethinkdb-2-1-5

Postgres by far. The inbuilt json support is great. In addition, Postgres as a whole has decades of bullet proofing behind it. MongoDB doesn't have many advantages, has many bugs and sharp edges, and doesn't have enough advantages to justify it's use. In [an old analysis by Jepsen](https://jepsen.io/analyses/mongodb-4.2.6), MongoDB 4.2.6 exhibited "read skew, cyclic information flow, duplicate writes, and internal consistency violations. Weak defaults meant that transactions could lose writes and allow dirty reads."

Will note that json/text columns are better than jsonb. Lower serialization overheads and surprisingly smaller binary size. Or better yet, just normalize to a columnar format.