Daniel Abadi

FaunaDB uses Calvin, a transaction protocol developed by Daniel Abadi. Their blog post explains it nicely, after a bit of a slow start:

https://fauna.com/blog/consistency-without-clocks-faunadb-transaction-protocol

But in summary:

1. A 'transaction' is a self-contained blob of code which reads input, does deterministic logic, and writes output (so not like a traditional RDBMS transaction, where the application opens a transaction and then interleaves its own logic between reads and writes)

2. When a transaction arrives, the receiving node runs it, and captures the inputs it read, and the outputs it wrote

3. The transaction, with its captured inputs and outputs, is written to a global stream of transactions - this is the only point of synchronisation between the nodes

4. Each node reads the global stream, and writes each transaction into its persistent state; to do that, it repeats all the reads that the transaction did, and checks that they match the captured input - if so, the outputs are committed, and it not, the transaction is aborted, and retried

The key idea is that because the process is deterministic, the nodes can write transactions to disk independently without drifting out of sync.

It's pretty neat. And it's exactly what Abadi wrote about a couple of months ago:

http://dbmsmusings.blogspot.com/2019/01/its-time-to-move-on-from-two-phase.html

This is also what VoltDB does (which Abadi worked on, along with Michael Stonebraker):

As an operational store,…

You are correct that they serialize cross-partition transactions. So if your workload has a reasonable percentage of write/update/delete operations it is possible that you will bottleneck on the global coordinator. From this blog https://fauna.com/blog/distributed-acid-transaction-performance, you can get 3300 transactions/second.
Daniel Abadi claims you can get to 500,000 trans/sec - http://dbmsmusings.blogspot.com/2018/09/newsql-database-systems-are-failing-to.html.

In any case, there is an open-source in-memory database called NDB (MySQL Cluster) that can get to 100 million write transactions per second. The echo chamber of SV has ignored it, but it's real:
http://mikaelronstrom.blogspot.com/2015/03/200m-reads-per-second-in-mysql-cluster.html

NDB has what i call lock-aware programming. When you read a row, you take either a read_lock or a write_lock, as it only supports READ_COMMITTED isolation level. But, it does not serialize cross-partition transactions. Transaction coordinators run independently, and it is up to the developer to ensure consistency by taking either read or write locks on rows.

That depends on what level of book, and what you want to learn from it. On "how real distributed systems are built", there isn't a really good book available in my opinion. I liked The Datacenter as a Computer, but it's scope is fairly narrow and is more focused on global infrastructure issues than software design issues. On the math behind distributed systems, Nancy Lynch's book Distributed Algorithms (http://www.amazon.com/Distributed-Algorithms-Kaufmann-Management-Systems/dp/1558603484) is my favorite, but it's getting to be quite out-of-date.

Absent a book, there are a lot of great resources online. High Scalability(http://highscalability.com/) has a good amount of quality links to distributed systems content, and some editorial content of varying quality. In general, it's a good site to follow if you're interested in how distributed systems are being used in industry. I also like to follow Aphyr's blog(http://aphyr.com/), Daniel Abadi's blog(http://dbmsmusings.blogspot.com/), and Henry Robinson's blog(http://the-paper-trail.org/blog/) and Peter Bailis's blog(http://www.bailis.org/).

That system works by giving up interactive transactions. But the foundationdb page claims to have interactive transactions, so it must be doing something else.

From Abadi's blog:

http://dbmsmusings.blogspot.com/2012/05/if-all-these-new-dbms-technologies-are.html

Calvin requires all transactions to be executed fully server-side and sacrifices the freedom to non-deterministically abort or reorder transactions on-the-fly during execution. In return, Calvin gets scalability, ACID-compliance, and extremely low-overhead multi-shard transactions over a shared-nothing architecture.

The comments on that post are pretty interesting, too.

And from the foundationdb features page:

http://foundationdb.com/#features

FoundationDB transactions are true interactive sessions, unlike distributed databases that require stored procedures. This means that client code can make an iterative series of reads and writes over the network to execute complex transactions.

A version which was rejected included the following (non-Apache-affiliated [afaik]) references:

https://www.xenonstack.com/insights/what-is-apache-arrow/

https://link.springer.com/chapter/10.1007%2F978-1-4842-1311-7_5

https://www.biorxiv.org/content/biorxiv/early/2016/08/23/071092.full.pdf

http://delivery.acm.org/10.1145/3110000/3103003/p138-Maas.pdf

https://www.theregister.co.uk/2016/02/17/apache_arrow_toplevel_project/

https://www.cio.com/article/3034279/big-data-gets-a-new-open-source-project-apache-arrow.html

https://www.infoworld.com/article/3033446/hadoop/apache-arrow-aims-to-speed-access-to-big-data.html

https://sdtimes.com/apache/guest-view-first-release-apache-arrow/

https://www.infoq.com/news/2016/12/le-dem-apache-arrow/

http://dbmsmusings.blogspot.com/2017/10/apache-arrow-vs-parquet-and-orc-do-we.html

https://dbmsmusings.blogspot.com/2017/10/apache-arrow-vs-parquet-and-orc-do-we.html

Hello, author here. That's a good question and something I had a hard time sorting out as I worked on this.

I think those fall into a different category confusingly sometimes called column-oriented databases. They're primarily used for analytic-focused tasks and get their name from storing data by column instead of by row (all data in a single column is stored to disk together).

I didn't include those as a separate category here because they're basically relational databases with a different underlying storage strategy to allow for easier column-based aggregation and so forth.

My colleague shared this article [1] with me, which definitely helped inform how I distinguished between the two in my head.

[1] http://dbmsmusings.blogspot.com/2010/03/distinguishing-two-major-types-of_29.html