
It's definitely true that people are writing more papers than they've ever had and there's nothing to stop that trend.
On the other hand, Albert Einstein was able to discover some amazing things with simple pencil-and-paper calculations. The last "big theory" in fundamental physics to be confirmed in the lab was the electroweak interaction and that was 1983. Many of the great physicists of our day, such as Ed Witten and Steven Hawking have never had a prediction confirmed in an experiment.
To be fair, there has been some excitement in experimental fundamental physics in the last few years. Neutrino oscillations are confirmed, there are glimmerings of dark matter, and something interesting might happen at the LHC. However, even if Superstring theory is true, it's possible that's there is very little new physics left to be discovered within reasonable reach.
Now, look at other fields and you find similar stories. There is a huge amount to discover in the field of biology, but there's never going to be another Darwin or Crick.
All fields of knowledge face limits that are caused by undecidability -- computer scientists know they can't solve the halting problem by static analysis, and the field of classical chaos has been dead for decades. There's a linear plane, and a handful of systems that can be solved exactly, each with a pertubative island around it, and then a vast undecidable ocean.
He touches on the career problems of people in physics, which is probably the field that's been the most pathological for the longest. When I got my PhD in 1998, the APS estimated that fewer than 3% of us would get permanent work in the field. I found it quite hard to find a professor in my department who didn't have at least one professor for a parent, so the odds for a bright kid who comes from a working class background and loves science are a lot worse than that.
For all we know we could be producing 10 Einsteins a decade but only keeping 2 in a lifetime.
It is John Horgan (and not Morgan) and the book is "The End Of Science: Facing The Limits Of Knowledge In The Twilight Of The Scientific Age".
The End of (one type of) Physics, and the Rise of the Machines
https://www.math.columbia.edu/~woit/wordpress/?p=10680
Seems there are two possible outcomes. The deluge of data leads to better correlation which smooths over the flaws in current models. And corrects errors with some minor fudge factor that contains no further significance.
From Dark Matter to Galaxies with Convolutional Networks
https://arxiv.org/abs/1902.05965
Or something deeply profound is discovered. The thing which cannot be ignored. And instead leads to an explosion of new physics. Recognizing patterns of the latter class will perhaps always be the domain of the human operator.
