Kris Nova, an engineer formerly at GitHub and Twilio, writes about Kubernetes, systems thinking, running Mastodon and engineering careers.
Commenters link her posts on Hachyderm infrastructure and organic versus mechanistic systems as useful reading on how complex, fragile software grows.
You may be interested in this article:
https://medium.com/@kris-nova/organic-and-mechanistic-systems-e576ac470845
What you are dealing with is organically grown software. No one really understands it, and it's likely to be fragile.
I agree with all those here that say the first thing to do is to introduce version control. You don't want to break that money machine without a way to revert.
Second, introduce some lightweight form of code review, but don't tick people off.
Third, if you want to do something like revise the user interface, consider adding an API to the existing organic code base, and build the new UI with that API. Generally, take that approach and avoid jamming something into the middle of the existing code that no one fully understands.
Very familiar sentiment to thoughts I have been thinking and feeling over the past year or so. Interesting to see more of this stuff popping up. I am starting to think unionizing is going to be the path to start pointing the majority of folks towards.
https://medium.com/@kris-nova/why-i-wrote-hacking-capitalism-5094d26d4fef
A great article published after I posted this, by Kris Nóva (GitHub, formerly Twilio). She self-hosts hachyderm.io, which currently supports 6,000 users:
https://medium.com/@kris-nova/hachyderm-infrastructure-74f518bc7472
Forward-looking Hachyderm infrastructure requirements: https://hackmd.io/qrR8obReTlaHkQXkHQNXJw