
I also found it impossible to rank books, so instead I took roughly the top 20, grouped them into themes, and then ranked the themes based on how important I think they are.
1. Deming is the primary writer of the type of management we need to be competitive: Deming, The New Economics & Out of the Crisis.
2. There is a wide variety of ways for humans to organise. Nobody is forcing the default capitalist bureaucracy on you, so be creative: Graeber, Bullshit Jobs & Dawn of Everything & The Democracy Project.
3. How people are motivated: Pink, Drive.
4. How to create a high-trust, less authoritarian, open, collaborative environment within a traditional hierarchy: Marquet, Leadership is Language.
5. Communication: Voss, Never Split the Difference. Faber, How to Talk So Kids Will Listen.
6. How to help organisations who don't want to admit what their problem is: Weinberg, Secrets of Consulting.
7. How to structure organisations and processes to deliver maximum value: Reinertsen, Principles of Product Development Flow. Ward, Lean Product and Process Development. Womack, Machine That Changed the World.
8. How to innovate cheaply: Westrum, Sidewinder.
9. How to estimate things properly: Hubbard, How to Measure Anything. Tetlock, Superforecasting. Savage, The Flaw of Averages.
10. How to train people to be experts at things: Hoffman, Accelerated Learning.
11. How people fail at decisions: Kahneman, Thinking, Fast and Slow & Noise.
I had to leave out a few about statistics, causality inference, project portfolio investment that didn't make the top 20 cut.
Ward's Lean Product and Process Development is also a good take on those ideas.
A lot of the things we build into our software is complexity nobody really needs. One way of managing it is by never making it in the first place. This takes at least two things:
- Thorough discussion with the customer about what problem they really are trying to solve. They will come to you with what they think is the solution. That way leads unnecessary complexity.
- Rough economical analysis of choices. Often we do things because that's what one ought to do. Make a quick napkin calculations. A lot of the things one ought to do are not actually necessary.
There are way too many books to recommend here, but perhaps some of the most relevant are
- Deming's The New Economics, and Out of the Crisis,
- Reinertsen's Principles of Product Development Flow,
- Ward's Lean Product and Process Development.
It's difficult to give a single recommendation, because it's a broad topic that branches off into several important directions. A continuously improving learning organisation takes many interlocking ideas to work, relating to long-term value powered culture, economic and probabilistic reasoning, uncertainty reduction techniques, including running experiments and gathering data on them, and analysing that data. But more important than running experiments are figuring out where the heck you stand and where you want to go. Just knowing that gets you halfway there.
----
These first four I would consider mandatory for anyone interested in this kind of stuff.
- Out of the Crisis or The New Economics (I keep confusing these with each other because I read them after each other -- well, you've probably read these already given your comment)
- The Machine That Changed the World (this seminal study introduced the industry to the idea of lean manufacturing, and it tells you exactly how important the ideas of lean/flow are)
- Principles of Product Development Flow (the author is a guru in applying lean/flow ideas to product development and teaches you to not blindly apply lean manufacturing ideas to development, but to adapt them to the economic conditions of development)
- Lean Product and Process Development (an opinionated but very intriguing view on how product development should be done, with focus on building a learning organisation)
----
These next ones are a little less mandatory and not necessarily extremely topical, but still interesting.
- How To Measure Anything (before you collect data the most expensive way possible -- with experimentation -- try estimation using proper techniques first; it's really cheap and surprisingly good)
- Understanding Variation (a good introduction to statistical process control, which helps you interpret your operational experience)
- Learning To See (value stream mapping is a way to visualise some wastes in processes)
- Accelerate (huge, impressive study of what high performing development organisations do differently)
- Drive (primer on the psychology of motivation)
- Trustworthy Online Controlled Experiments (there are probably better books on experimental design -- I've been eyeing some of Deming's myself -- but this one is probably as good a primer as any)
- The Flaw of Averages (to get more accurate predictions, focus on distributions and ranges, not single numbers)
- Toyota Kata (a rote way to build a culture of continuous improvement)
- Making Software: What Really Works, and Why We Believe It (general high-quality exposition on evidence-backed techniques -- maybe not strictly in the scheduling/project management realm)
- Rapid Development (like Making Software except older and thicker)
- Turn the Ship Around (case study in applying some of these principles to completely change the performance of a team)
- Lean Enterprise (more on how to adapt lean ideas to the whole development organisation, with case studies, IIRC)
- Willful Ignorance (what are probabilistic statements, really?)
----
I have many more recommendations if you want to go deeper into a specific sub-topic, but this is probably enough for a cursory comment.
Functional departments and their managers are responsible for creating reusable knowledge. A project leader (chief engineer, in Toyota speak) is responsible for delivering value in a particular project.
The project leader accomplishes their goals by help from the functional departments. However, the project leader has no formal authority over the engineers in the functional departments.
In other words, there will be a constant negotiation between the project leader who argues for something to be designed for a project, and the functional department which must be convinced it's worth spending time to do that thing.
The way a Toyota executive put it was "A great car is made from lots of conflict".
I'm not doing this approach justice in my description. I suggest just reading Lean Product and Process Development by Allen Ward. Lots of good ideas in it.
EDIT: I figured it out. For anyone else who is confused, Allen C. Ward wrote Lean Product and Process Development, and the parent is saying that Ward is simply repackaging what W. E. Deming taught.