Designing Data-Intensive Applications
Data is at the center of many challenges in system design today. Difficult issues need to be figured out, such as scalability, consistency, reliability, efficiency, and maintainability. In addition, we have an overwhelming variety of tools, including relational databases, NoSQL datastores, stream or batch processors, and message brokers. What are the right choices for your application? How do you make sense of all these buzzwords? In this practical and comprehensive guide, author Martin Kleppmann helps you navigate this diverse landscape by examining the pros and cons of various technologies for processing and storing data. Software keeps changing, but the fundamental principles remain the same. With this book, software engineers and architects will learn how to apply those ideas in practice, and how to make full use of data in modern applications. Peer under the hood of the systems you already use, and learn how to use and operate them more effectively. Make informed decisions by identifying the strengths and weaknesses of different tools. Navigate the trade-offs around consistency, scalability, fault tolerance, and complexity. Understand the distributed systems research upon which modern databases are built. Peek behind the scenes of major online services, and learn from their architectures
Stats for nerds
457recommend
846mentions
659people
2014–2026years
67% recommend32% neutral1% critical
Mentions per year
What Hacker News says
- There is a whole chapter in Designing Data Intensive Applications dedicated specifically to handling atomic transaction concurrency issues while avoiding the performance trade-off you're describing here, you will definitely find it interesting.
https://www.oreilly.com/library/view/designing-data-intensiv...
- I recently read DDIA for the first time, and I was surprised by how enjoyable it was after reading AI slop for a year.
- A Critique of the CAP theorem by Martin Kleppmann.
https://arxiv.org/abs/1509.05393
Basically naive CAP theorem assumes "no latency (delay) allowed", in which case, yeah, sure, the quorum must agree or you are writing to a split-brain quorum somehow. But what if delay sensitivity is a knob that can be tuned, and writes have conflict resolution?
Martin Kleppmann's book Designing Data-Intensive Applications is considered a useful tome for anyone who needs a insight in the complexity involved in distributed systems (hint: getting data in ram flushed to an SSD also involves multiple moving parts)
- Books that come into my mind that might be useful to you:
A Philosophy of Software Design — John Ousterhout
Designing Data-Intensive Applications — Martin Kleppmann
Domain-Driven Design — Eric Evans
There might be some courses as well, but I never looked into them.
This also works quite well:
Write down the problem. Think real hard. Write down the solution. (Thank you, Feynman).
- DDIA is great. I read and re-read the small section about LSM merge tree in the first and second edition of the book and got interested in this stuff. It is always a pleasure to read about internals.
- Blogs are significantly more up to date with most technical topics. Books focused on specific technologies by contrast go out of date as soon as they are published. For example: https://simonwillison.net/ is HN's favorite blog this year.
Evergreen computer science topics don't go out of date though. (example of evergreen: https://dataintensive.net/)
Related
The architecture of open source applications
204 recommend · 257
Clean Code
298 recommend · 951
A Philosophy of Software Design
163 recommend · 342
Clean Architecture
47 recommend · 142
The algorithm design manual
195 recommend · 321
Crafting Interpreters
295 recommend · 521
