Decentralized Thoughts is a group blog by distributed computing researchers on consensus, Raft, synchrony models, and cryptographic protocols.
Commenters recommend it in blog lists and link its posts on Raft liveness problems and Ittai Abraham's series on distributed system models.
Hi all! I just wanted to say thanks for the awesome questions. I can't say I have the answers to these questions, but I can lay out some of my intuition and would love to work together with anyone that also finds these questions interesting!
1. Will this scale?
Blockchains vary from the order of ~10^0 tps (PoW) to ~10^4 tps (PoS).
https://decentralizedthoughts.github.io/2019-06-23-what-is-the-difference-between/
Wikipedia sees 50K HTTP requests/second and 80K SQL queries/second.
https://www.datacenterknowledge.com/uptime/data-center-providers-adjust-doing-business-remotely
But it is also read-heavy. Let's say if 10% of the workload includes writes (probably too conservative), we barely squeak by... I'm willing to guess the number is way lower than that. They saw 10M page edits for the entire month of Dec 2018.
https://stats.wikimedia.org/EN/TablesDatabaseEdits.htm
My guess is, this might be doable :-D
2. Who runs this? and who pays?
Great question. The blog completely skips the question about incentives which in my opinion will vary depending on the app. For creating an censorship-resistant Wikipedia, it'll be different than a communications app. Some apps might be better off permissioned (e.g. enterprise settings)
Wikipedia is 43GB of text and 23TB of rich media (images and video).
https://en.wikipedia.org/wiki/Wikipedia:Size_of_Wikipedia
If we limit ourselves to just Wikipedia text, it might be reasonable to do this permissionless!
Hoping to write a blog post…
I have many recommendations of different kinds:
## Blogs:
- http://muratbuffalo.blogspot.com/
- https://bartoszsypytkowski.com/
- https://decentralizedthoughts.github.io/
- https://www.the-paper-trail.org/
- https://blog.acolyer.org/
- https://pathelland.substack.com/
## Other web resources
- https://aws.amazon.com/builders-library/ - set of resources from Amazon about building distributed systems
- https://www.youtube.com/playlist?list=PLeKd45zvjcDFUEv_ohr_HdUFe97RItdiB - lecture series from Cambridge
## Books
- https://www.cl.cam.ac.uk/teaching/1213/PrincComm/mfcn.pdf - A great book on the maths of networking (probability, queuing theory etc...)
Paxos shouldn't be the first protocol you learn -- I think it gives consensus this mistaken reputation for being "difficult to understand".
Consensus protocols (and the related problems of state machine replication, permissioned blockchains) have really come a long way over the years. They've become more performant and much simpler (in my opinion).
(I'll take the opportunity to pub streamlet: it's simpler, has only two message types "propose" & "vote", and it tolerates malicious faults. https://decentralizedthoughts.github.io/2020-05-14-streamlet/)
But raft isn’t strongly consistent, it has known liveness issues.
https://decentralizedthoughts.github.io/2020-12-12-raft-liveness-full-omission/