I’m mid career but worked out a couple of decades ago how crazy the evidence based medical system is.
As a doctor, on the one hand you’re genuinely trying to do the right thing by the patients, on the other hand you’re trying not to get sued. Unfortunately they are not the same thing although of course they should be.
There is a crazy industry now of accreditation, evidenced based medicine and conferences. It feeds off itself and generates billions of dollars annually around the world.
One way of working out best practice is simply a straw poll of experts. I went to conference once where the speaker put up a series of clinical scenarios with options a,b,c,d how to treat. The audience chose via their phone which option they’d choose. Most scenarios had responses of about 70-80% for one option and an even distribution of the other 3.
Although a couple had 90+% one answer and other scenarios had a 4 way even spread.
Everyone practiced in the field and had read the evidence, thus simply asking a group of clinicians what they actually do was heading towards the best answer.
It’s nowhere near perfect but is the best way to summarise opinion. The rest of the time you’re doing what you think’s right based around your exposure to evidence (including your own practice) and available resources.
And if you practice long enough then you’ll likely see a total reversal of practice for a certain condition throughout your career.
The bounds of your practice are limited by “trusted third parties” who define what’s accepted practice, based around their own interpretation of evidence. All good and we’ll until you get a bit older and those people on those regulatory bodies are your peers...then you start to think what does that dufus know?
As an example, I personally know the head of ATAGI here in Australia. He’s not a dufus but he obviously can’t see much wrong with the evidence for the vaccines, contrary to what many commentators claim. I’ve resisted calling him but he’s just like me, nothing particularly special but now he’s making massive decisions
Hanin (2017) is profoundly naïve when writing, "1. Clinical trials should be publicly funded and conducted by biomedical researchers, medical doctors and statisticians with no relation to industry and no conflicts of interest."
Publicly funded research may also have significant relation to industry and conflicts of interest (most often, political, but others as well). Arguably, it will tend to be much worse than industry (the extreme case being Lysenkoism).
Kealey has written plenty about how public funding of science is uncorrelated with economic growth, implying that there are deeper issues than the funding sources.
My unscientific guess is that failures of science are just a symptom of the failures of the overall system. So even harder to fix. But Lykoudis is a great example of how truth can win out _despite_ the corruption of the system.
You just have to look at the history of medicine, to see that the biggest leaps forward were made by people who were iconoclasts, able to think outside the square and had real understanding and vision. They were all sidelined, pilloried, subjected to inquisitions from institutions and usually died paupers or in mental assylums. A lot committed suicide, like Dr Louis Pillemer (properdin).
Great overview of the issues. As an appendix I would add Sander Greenland (115k citations, epidemiology textbook author) to the recommended reading list. He is often commenting on dishonest use of stats in medicine. Here is an example (with not enough views): https://www.youtube.com/watch?v=kOV3UzJIKDw
My daughter has long covid, or at least she changed from a healthy quite self confident 33 year old woman to someone that is anxious and seeks shelter. Reading up on studies and medical information about long covid and related maladies like ME I am flabbergasted how utterly confused the field seems. I thought this mess was just limited to nutritional science but the rot seems wide spread.
It is telling that the consensus of comments is that EBM is not a unique problem, but rather is the prow of the ship of managerial dictatorship. The brilliant evolutionist Bill Hamilton prophesized the future of human society as more or less ant-like. In my view, the only way to avoid a return to the Sumerian societal model is to ditch democracy and replace it with demarchy through sortition. Unfortunately, that revolution cannot be carried out by a morally degraded populace like the USA's. We are just too lazy.
Evidenced based teaching is no different. The result is so evident when you go to a store or restaurant expecting some semblance of customer service from a frightening blue haired young adult with air pods dangling from their ears. I don't see a more intellectually competent society coming from government schools over the past 30+ years. I see people who put diapers on their face, hid the basement and injected who knows what into their bodies because they were told to. Many have turned their backs on the God that created them and worship at the alter of "the science" while self medicating with potent, dangerous but "legal" intoxicants.
I'm getting a sense that "evidence based" whatever is codename for "managerial takeover of" that very thing. Medicine was ground zero, and other fields are following on that track.
"Tyranny of Metrics" - Deming did it better in "Out of the Crisis". Both talk about misuse of metrics, I though Deming had the better discussion, telling a fix for each bad example.
Neither addressed the intentionally fraudulent use of metrics.
Using the title "Tyranny of Metrics" seems to blame metrics for the problems, not the user thereof, which was off-putting.
Chapter 14 of Tyranny covers transparency in government, of which we apparently have too much. The book was published after Pelosi had said "We have to pass the bill to see what's in it". That is enough of a brain fart to stink up the whole book for me.
Agreed that the book could be better. I find it a good and fairly modern introduction to goodhart's law for people who haven't delved too deeply. But I also agree that the rabbit hole goes far deeper.
I’m mid career but worked out a couple of decades ago how crazy the evidence based medical system is.
As a doctor, on the one hand you’re genuinely trying to do the right thing by the patients, on the other hand you’re trying not to get sued. Unfortunately they are not the same thing although of course they should be.
There is a crazy industry now of accreditation, evidenced based medicine and conferences. It feeds off itself and generates billions of dollars annually around the world.
One way of working out best practice is simply a straw poll of experts. I went to conference once where the speaker put up a series of clinical scenarios with options a,b,c,d how to treat. The audience chose via their phone which option they’d choose. Most scenarios had responses of about 70-80% for one option and an even distribution of the other 3.
Although a couple had 90+% one answer and other scenarios had a 4 way even spread.
Everyone practiced in the field and had read the evidence, thus simply asking a group of clinicians what they actually do was heading towards the best answer.
It’s nowhere near perfect but is the best way to summarise opinion. The rest of the time you’re doing what you think’s right based around your exposure to evidence (including your own practice) and available resources.
And if you practice long enough then you’ll likely see a total reversal of practice for a certain condition throughout your career.
The bounds of your practice are limited by “trusted third parties” who define what’s accepted practice, based around their own interpretation of evidence. All good and we’ll until you get a bit older and those people on those regulatory bodies are your peers...then you start to think what does that dufus know?
As an example, I personally know the head of ATAGI here in Australia. He’s not a dufus but he obviously can’t see much wrong with the evidence for the vaccines, contrary to what many commentators claim. I’ve resisted calling him but he’s just like me, nothing particularly special but now he’s making massive decisions
Hanin (2017) is profoundly naïve when writing, "1. Clinical trials should be publicly funded and conducted by biomedical researchers, medical doctors and statisticians with no relation to industry and no conflicts of interest."
Publicly funded research may also have significant relation to industry and conflicts of interest (most often, political, but others as well). Arguably, it will tend to be much worse than industry (the extreme case being Lysenkoism).
Kealey has written plenty about how public funding of science is uncorrelated with economic growth, implying that there are deeper issues than the funding sources.
My unscientific guess is that failures of science are just a symptom of the failures of the overall system. So even harder to fix. But Lykoudis is a great example of how truth can win out _despite_ the corruption of the system.
Agreed. People reach for "the state" as the answer because it's always top of mind. Sadly, it only makes things worse.
You just have to look at the history of medicine, to see that the biggest leaps forward were made by people who were iconoclasts, able to think outside the square and had real understanding and vision. They were all sidelined, pilloried, subjected to inquisitions from institutions and usually died paupers or in mental assylums. A lot committed suicide, like Dr Louis Pillemer (properdin).
Great overview of the issues. As an appendix I would add Sander Greenland (115k citations, epidemiology textbook author) to the recommended reading list. He is often commenting on dishonest use of stats in medicine. Here is an example (with not enough views): https://www.youtube.com/watch?v=kOV3UzJIKDw
Really captivating webinar you linked to, Erik.
Thank you very much.
Your evidence is compelling. Metrics may have a place, but never, ever the most important or telling place.
My daughter has long covid, or at least she changed from a healthy quite self confident 33 year old woman to someone that is anxious and seeks shelter. Reading up on studies and medical information about long covid and related maladies like ME I am flabbergasted how utterly confused the field seems. I thought this mess was just limited to nutritional science but the rot seems wide spread.
Interesting.
Peter C. Gøtzsche and Christine Stabell Benn are both from Denmark.
And Denmark had probably the most sane vaccination policy in the West.
https://world.hey.com/dhh/a-pandemic-dispatch-from-denmark-80dc7aca
I wonder if these two things are related.
It is telling that the consensus of comments is that EBM is not a unique problem, but rather is the prow of the ship of managerial dictatorship. The brilliant evolutionist Bill Hamilton prophesized the future of human society as more or less ant-like. In my view, the only way to avoid a return to the Sumerian societal model is to ditch democracy and replace it with demarchy through sortition. Unfortunately, that revolution cannot be carried out by a morally degraded populace like the USA's. We are just too lazy.
Evidenced based teaching is no different. The result is so evident when you go to a store or restaurant expecting some semblance of customer service from a frightening blue haired young adult with air pods dangling from their ears. I don't see a more intellectually competent society coming from government schools over the past 30+ years. I see people who put diapers on their face, hid the basement and injected who knows what into their bodies because they were told to. Many have turned their backs on the God that created them and worship at the alter of "the science" while self medicating with potent, dangerous but "legal" intoxicants.
I'm getting a sense that "evidence based" whatever is codename for "managerial takeover of" that very thing. Medicine was ground zero, and other fields are following on that track.
If you haven't run into it yet, check out the book "Turtles All The Way Down: vaccine science and myth". It's a worthy addition to your list. https://www.amazon.com/Turtles-All-Way-Down-Vaccine/dp/9655981045
"Tyranny of Metrics" - Deming did it better in "Out of the Crisis". Both talk about misuse of metrics, I though Deming had the better discussion, telling a fix for each bad example.
Neither addressed the intentionally fraudulent use of metrics.
Using the title "Tyranny of Metrics" seems to blame metrics for the problems, not the user thereof, which was off-putting.
Chapter 14 of Tyranny covers transparency in government, of which we apparently have too much. The book was published after Pelosi had said "We have to pass the bill to see what's in it". That is enough of a brain fart to stink up the whole book for me.
Agreed that the book could be better. I find it a good and fairly modern introduction to goodhart's law for people who haven't delved too deeply. But I also agree that the rabbit hole goes far deeper.