Strange Loop Conference
12 HN recommenders · recommended in 19 months, 2014–2024
40:20 · 56K views · Sep 20, 2014
Will Wilson of FoundationDB explains how the team tested their distributed database using deterministic simulation. By running the whole system in a simulator with injected faults and reproducible seeds, they could find rare concurrency and network bugs.
HN treats it as the canonical talk on simulation testing and a mind-blowing example of how far rigorous testing can go.
20 comments from Hacker News
Soooo stoked. This should make futzing with it to get started a lot easier. I've been thinking about going to the FoundationDB summit to hopefully learn more... Does anyone have any good resources for learning about it? I've tried a few times but find the documentation a bit hard to grok. Maybe I'm just slow. Also if you haven't seen it, this is one of the most amazing talks on distributed systems (because it's about testing them) I've ever seen (by Will Wilson from FoundationDB pre-Apple merger): this video Truly mind blowing how good FoundationDB is supposed to be for its use cases, and this Mongo layer is just the business when it comes to helping adoption.
"FDB can tolerate f failures with only f+1 replicas." Wait a minute, I know that formula... Viewstamped Replication?? I need to read the foundation DB paper. (I mainly read CRDT stuff so hopefully it's understandable). --- In general I'm really impressed foundation DB folks. The talk "Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson blew my mind. TL;DR they spent the majority of their initial dev effort into making a simulation of the database, then when they were happy with that plugged in real storage, time and networks at the end. Well worth a watch for anyone interested in distributed systems & reliability. this video
This talk was about Foundation DB was brought up recently, and it's pretty amazing. I recommend watching the whole thing, but to be brief they are taming the "infinite interleavings" problem through determinism. "Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson this video They wrote an interesting Actor DSL that compiles to C++ and is completely deterministic, and they torture this deterministic engine with generated test cases on a cluster every night. I guess you could say that the whole cluster is necessarily non-deterministic, but an individual node is deterministic, given an ordering of the messages it receives.
One of their engineers gave a great talk at Strangeloop ( this video ) that shows some of the amazing simulations they ran FoundationDB through. It's well worth a watch if you want to understand the punishment that they put this database through to make it stand out.
Here's another excellent talk at Strangeloop on FoundationDB's simulation testing by Will Wilson in 2014: this video