
"An Aristotelian Realist Philosophy of Mathematics: Mathematics as the Science of Quantity and Structure" by James Franklin
"German Science" by Pierre Duhem
You must realize that proof and empirical result are never a replacement for common or good sense. The very attitude that proof is self-sustaining is as silly as the idea of being able to pull yourself up by your own bootstraps. I encourage you to read Duhem's "German Science" in which he describes exactly the silliness of "science" when it is divorced from common sense. It's not magic, my friend. A man of little good sense will lack a sound place to begin reasoning from. Indeed, a man completely devoid of common sense can only begin by reasoning from arbitrary postulates. However, no amount of rigor can ever make up for the initially poorly chosen postulates. Theorems are only as true as their axioms. As Pascal once said, principles are intuited, propositions inferred.
Common sense is not infallible, of course, and certainly science can help us refine it. But common sense is the necessary starting point of all science (the ancient Greeks talk about doxa and endoxa). Without it, there would be no starting material for your empirical science or your proofs. There would be no interpreting of the results of your experiment or any understanding of what the experiment should consist of in the first place.
Of course, you must also be careful to understand that not everyone possesses a good deal of common sense even though they may appeal to it. Common sense is not the same as popular opinion.
Science isn't a magical thing that is somehow divorced from "everyday reality" or some gnostic religion that tells you how it really is without being somehow derived from experience.
I'm reminded of Duhem's rather humorous "German Science" where he characterizes German mathematics as being disproportionately oriented toward rigorously deducing nonsense from arbitrary axioms as opposed to the French tendency to rely on intuition to arrive at sensible axioms but neglecting systematic rigor afterwards.