Luke Oakden-Rayner is a radiologist and deep learning researcher who writes about medical AI, imaging datasets and evaluating machine learning in healthcare.
Commenters recommend it as a skeptical, informed view on machine learning in medical imaging, since he is a radiologist first; posts on doctors versus machines and Covid CT are cited.
It's not really a doctor shortage, more like a service shortage: https://www.nytimes.com/2016/11/08/upshot/a-doctor-shortage-lets-take-a-closer-look.html
Generally, there's an ongoing reformation of the system. From the perspective of medical insurance, the current medical system is actually too lax, in that it certifies and retains doctors who get frequent malpractice suits. http://digitalcommons.law.umaryland.edu/cgi/viewcontent.cgi?article=1174&context=jhclp http://www.medscape.com/viewarticle/833141
The solution to that seems like better data on health care (utilization, treatment outcomes, etc.) and accessible analytics (databases for professionals, patients, providers, etc.) and automation of complex/risky procedures/tests (deskilling). But those take time and effort and investments, which have been made but aren't finished/stable yet (somewhere in the middle of Gartner's hype cycle). http://sloanreview.mit.edu/case-study/when-healthcare-gets-a-healthy-dose-of-data/ http://www.webmd.com/health-insurance/using-doctor-ratings-sites https://lukeoakdenrayner.wordpress.com/2017/05/03/the-end-of-human-doctors-understanding-automation/
> What would it take to make it 290?
Once people start trusting the system again (which they don't, currently: http://www.hhnmag.com/articles/7802-the-trust-chasm-in-healthcare-analysis-of-the-2016-trust-survey) it might be possible to charge more for services and hence employ more people. But they wouldn't be physicians, the generic trend…
It might be helpful for folks to look at the blog post written by one of the authors:
https://lukeoakdenrayner.wordpress.com/2021/08/02/ai-has-the-worst-superpower-medical-racism/
or the paper itself
https://arxiv.org/pdf/2107.10356.pdf
I see a lot of "oh it's probably just picking up on x y z" when x, y, and z are things they explicitly checked for:
1) "It's probably just the names or other metadata" – they only gave it pixel data to train on. To control for things like metadata overlaid on the image (e.g., a name written on the image) they divided the images into 3x3 sections and trained classifiers on each section separately.
2) "It's probably some artifact of how the hospital marked up the images" – they used something like 7 different datasets from different hospitals and different modalities (X-Ray and CT).
If it is cheating somehow, it's not doing it in an obvious way that you can think of in a minute or two. Also note that they had more than just medical folks working on the paper; the author list includes plenty of computer scientists. It's unlikely they're making an elementary ML mistake here.
Hi, I'm the author of the blogpost (didn't expect to see it here!) and I largely agree with you.
I think I would say that the use of private datasets isn't actually a problem though, the problem is that the methods and analysis in many of the papers is horribly flawed. I could reasonably trust a paper with good methodology - all of medical research is on private data essentially. But the majority is, as you say, pure hype.
I actually wrote another blog post a bit back that was popular on some simple rules to separate hype from reality in medical AI work - https://lukeoakdenrayner.wordpress.com/2016/11/27/do-computers-already-outperform-doctors/
Everyone swooning with optimism over this result should the machine learning reddit comments on it first.
https://old.reddit.com/r/MachineLearning/comments/ehpllt/deep_learning_model_for_breast_cancer_detection/
And also linked blog[edit]:
https://lukeoakdenrayner.wordpress.com/2017/12/06/do-machines-actually-beat-doctors-roc-curves-and-performance-metrics/
TL;DR; There are sooo many subtlties to stuff like this that this things really shouldn't be taken at face value. This is far from replacing doctors in anything.
I found the Statistical Learning self paced course on Stanford's site to be a great formal intro to ML algorithms implemented in R, and it is taught by the inimitable Hastie and Tibshirani: http://statlearning.class.stanford.edu
This post on ML in medicine is a pretty good overview of everything that has been going on recently and the nuances often lost in the current hype: https://lukeoakdenrayner.wordpress.com/2016/11/27/do-computers-already-outperform-doctors/
For another perspective on applying machine learning to medical imaging, I recommend the blog of Luke Oakden-Rayner[1]. He's a radiologist first, so he's in a great position to bring some well-needed skepticism to the conversation. I learned about a lot of complications that I never would have imagined as a lay-person.
[1] https://lukeoakdenrayner.wordpress.com/