This is a public peer review of Scott Alexander’s essay on ivermectin. You can find an index containing all the articles in this series here. If I’ve learned one thing in this whole ordeal, it’s that we’re looking at an arms race. Peer review was instituted as a way to guarantee quality, but peer reviewers don’t have time to actually reconstruct papers, so their comments are mostly superficial. A well-written paper (perhaps by a dedicated
The behaviour of the medical community, and some of those to whom you have drawn attention in this essay, was eye-opening, to say the least, and not in a good way.
Very pleased that you have proposed a better alternative to the current, failing, model.
"Modern academic publishing has many mechanisms of protecting people’s egos, and very few mechanisms of encouraging, or at least theoretically allowing, corrections to materials that are out there."
This has always astonished me. Such an obvious flaw that has never been corrected, and yet the web would allow us to create very effective mechanisms for this
I know of studies with major, major flaws that remain well cited because knowledge of those flaws only exists in a few heads and not on paper.
Fisher introduced randomization to try to eliminate the effects of potential covariates on the independent variable in an agricultural experiment. There is typically not much variation in moisture, sunlight, and mineral nutrients across a few thousand square meters of agricultural soil. Similarly in a laboratory environment, variation can be kept low. However, physiological, immunological, psychological, and even anatomical variation exists among human subjects in complex ways and to extents that make medical RCTs potentially fallible unless sample sizes are very large. RCTs are more like a silver standard because they can become tarnished, through bad luck or through intent. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5653589/#:~:text=Randomized%20controlled%20trials%20have%20become,a%20mathematician%20in%20Great%20Britain.
Regarding opening peer review to the public, two different venues should be considered separately: 1) a forum for interested parties to evaluate the credibility and worth of the research publicly, and 2) public reviews by reviewers appointed by the journal to determine acceptance/rejection. Both would improve science, but it seems wise to distinguish between "interested persons" and "appointed peer reviewers" so as not to give those favoring the old-boy system a strawman to use.
You're on fire. I'm going to incorporate some of your thinking on consilience into my eventual essay on handling skepticism... good stuff.
Great work Alexandros.
The behaviour of the medical community, and some of those to whom you have drawn attention in this essay, was eye-opening, to say the least, and not in a good way.
Very pleased that you have proposed a better alternative to the current, failing, model.
Great article! Thanks for the laugh at the end "So these are my two main scientific takeaways:"... and you list three (3) 😁
PART 19 WHAT
"Modern academic publishing has many mechanisms of protecting people’s egos, and very few mechanisms of encouraging, or at least theoretically allowing, corrections to materials that are out there."
This has always astonished me. Such an obvious flaw that has never been corrected, and yet the web would allow us to create very effective mechanisms for this
I know of studies with major, major flaws that remain well cited because knowledge of those flaws only exists in a few heads and not on paper.
Fisher introduced randomization to try to eliminate the effects of potential covariates on the independent variable in an agricultural experiment. There is typically not much variation in moisture, sunlight, and mineral nutrients across a few thousand square meters of agricultural soil. Similarly in a laboratory environment, variation can be kept low. However, physiological, immunological, psychological, and even anatomical variation exists among human subjects in complex ways and to extents that make medical RCTs potentially fallible unless sample sizes are very large. RCTs are more like a silver standard because they can become tarnished, through bad luck or through intent. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5653589/#:~:text=Randomized%20controlled%20trials%20have%20become,a%20mathematician%20in%20Great%20Britain.
Regarding opening peer review to the public, two different venues should be considered separately: 1) a forum for interested parties to evaluate the credibility and worth of the research publicly, and 2) public reviews by reviewers appointed by the journal to determine acceptance/rejection. Both would improve science, but it seems wise to distinguish between "interested persons" and "appointed peer reviewers" so as not to give those favoring the old-boy system a strawman to use.
You are just so good. Thank you.
I want to know why bill gates was meeting with epstien to get a nobel prize. Im not sure where to start on that one.