
Please look up the papers on planning & quality & cost & schedule from Capers Jones. The textbook is literally titled "The Economics of Software Quality". Tom DeMarco's classic is "Controlling Software Projects".
I'm trying to give you a starting point to see that it's not even my anecdotes, but repeatable studies that show this.
Your examples are that requirements change, and it's hard. Of the few buildings I have built, the requirements changed drastically there in all cases, so I can only say that software engineering is not magically different. After decades of practice with software engineering my experience of planning lines up with the textbooks you can find on the topic, and not this blog post.
In it you'll find that this level of requirements gathering, blueprint/spec writing and code inspection are actually cheaper in a relatively short time (months, not years).
Writing specifications that are extremely detailed ends up giving you all the logical failures ahead of time, and makes typing code more of a spell checking exercise. By having the logical failures before the code ever runs, you get to debug & fix them before any code runs. This ends up being faster & cheaper in total than waiting for users to find bugs.
However - I understand that for many companies they actually don't care if users find bugs as time to market is more expensive than the future bug finding. But if you have a product already in the market, and you expect to support it for ~year, you probably can do much more detailed design and avoid so much pain.
* Code Complete - McConnell
* The Economics of Software Quality - Capers Jones
* Facts and Fallacies of Software Engineering - Robert Glass
* Making Software - Oram & Wilson
... and there are many others. Not all references have survived the test of time, L. Bossavit describes in his book " Leprechauns of Software Engineering" how several software development "facts" are in fact myths.
The approach you suggest suffers from a chicken & egg problem because you first have to be experienced enough to be able to evaluate such reference-free content, as the blog written by Martin Fowler. It's precisely the inexperienced that are likely to internalize such advice and perpetuate personality cults.
Your objections are invalid; one does not need to be better at a task than an individual in order to criticize them. Asking for references and proof is perfectly legitimate, at least if we want to approach our discipline more as engineering and less as "Strunk & White" as someone comically suggested in another comment.,
This is only one of these "must-read book list", what you can do is search several of them and see which ones are quoted by almost everyone!
About "everything" I would rate the following books very highly, for their sheer usefulness:
- Thinking Fast and Slow (really do the exercises!)
- then Neuroleadership which describes the purpose of brain structures, it's somehow wider than kahneman
- A Mind for Numbers (include very useful data points for knowledge workers)
- Distinction: A Social Critique of the Judgement of Taste
- Cutting Through Spiritual Materialism
About programming I would rate the following books highly, but at the same time it seems a bit fruitless to me to read about programming, rather than spending more time reading about psychology or sociology:
- the Economics of Software Quality
- Peopleware
- Pragmatic programmer
- Mythical Man month
^you'll see that these books are hardly kept secret, it's just that people quote them but don't actually read them, or so it seems.
Reading it is like the first time you run a profiler and you realize that what you think you knew is true in part, but that other factors you'd not counted are have an outsize effect that you've misestimated.
To this day, I know of no study that was able to demonstrate superiority of statically-typed languages - [1].
What studies clearly show, is that both in terms of productivity [2] and bug reduction [3], expressivity reigns supreme.
And Clojure is the most expressive [4] out of languages that can leverage huge ecosystems (Java and JS, soon C++ through Jank dialect).
[1] https://danluu.com/empirical-pl/ [2] PBX study from Economics of Software Quality by Caper Jones [3] https://arxiv.org/pdf/1901.10220 [4] https://redmonk.com/dberkholz/2013/03/25/programming-languag...