The Potemkin Argument, Part 20: Scott's Sociological Takeaway—Or—The Bret and Pierre Special
The Potemkin Argument, Part 20: Scott's Sociological Takeaway—Or—The Bret and Pierre Special
This is a public peer review of Scott Alexander’s essay on ivermectin. You can find an index containing all the articles in this series here.
We’ve reached the part of the essay where Scott is ready to tell us how he really feels about the various prominent characters in this saga.
The Sociological Takeaway
I’ve been thinking about this one a lot too.
Ivermectin supporters were really wrong. I enjoy the idea of a cosmic joke where ivermectin sort of works in some senses in some areas.
Let’s rewind a little bit here. Scott has run a meta-analysis, removing studies explicitly “erring towards severity.” When using standard analysis methods, the result is—in Scott’s words—a “clear” benefit for ivermectin, which he has acknowledged. In technical terms, the result is a statistically significant effect on the order of 55% or 29%, depending which of the two analyses you use.
Guess a maybe-kinda-sorta correlation with Strongyloides prevalence that doesn’t quite hold up to scrutiny and a funnel plot that drives unjustified conclusions is all we need to flatten the evidence curve.
You Come at the King, You’d Best Not Miss
But the things people were claiming - that ivermectin has a 100% success rate, […]
Would be nice if Scott gave us a source for that, but since he doesn’t, I’ll have to make an educated guess. When people bring up this point and are asked to cite it, they usually point to a quote from Bret Weinstein, which is either this:
…or this:
I'm unvaccinated but I am on Prophylactic Ivermectin, and the data—actually shocking as this will be to some people—the data suggests that Prophylactic Ivermectin is something like a 100% effective at preventing people from contracting COVID when taken properly.
Both quotes are from around that same time—early-mid summer 2021. While we can debate the particular quotes and qualifiers, I’ll note that in both cases Bret Weinstein is clearly talking about prophylactic use of ivermectin, not early treatment.
By late summer 2021, Bret Weinstein sought out the data from Carvallo—and not seeing the detailed data he expected—said the following (in the same podcast the original comment was made):
So, pretty much every person on the internet who said that prominent ivermectin proponents claimed it provides 100% protection, basically latched on to one or two comments by a single proponent who—after helping to demonstrate that the study those quotes were based on was less than trustworthy—declared that the study should be now given zero evidentiary value. This makes me trust him more, not less. I think this position is pretty obvious.
Now, given that Scott reviewed studies that did find positive effects for early treatment—and we finally do understand that earlier treatment is better—are we ready to accept that prophylactic treatment is basically the optimal case of early treatment? Where the drug is circulating when the virus first lands, and therefore a result in the high 80s or 90s is completely plausible?
It blows my mind that Scott reviewed early treatment studies, concluded (after correction, but still) that there is a “clear” signal of efficacy, and then the essay still talks about 100% success (at what?) as a way to frame the supporters as having been wrong. Why not address the claims of someone such as Dr. Tess Lawrie, whom every other main proponent defers to as analytically superior, anyway?
Remember kids: the golden rule of argumentation is to argue against others as you would have them argue against you. While we can discuss why those statements were made at the time, and the fact that they were later retracted—as opposed to the many statements we’ve discussed so far that remain intact (until now, at least)—the most important insight is that they’re not really arguments. The claim is that “someone on your side said something wrong at some point,” but who in their right mind would argue with that claim? And somehow, Scott fails to even support that layup of a claim, given that his research doesn’t actually address the statements he most likely is trying to discredit.
that you don’t need to take the vaccine because you can just take ivermectin instead […]
I don’t normally do this, but in this case, I’ll make an exception: I assume when it comes to vaccines, Scott will not be quoting all the times all the top public health officials said it was “virtually 100% efficacious,” so I’ll do it instead:
This is not an argument against vaccines. This is an argument against silly ways to argue your case. Just say no to the dark arts.
[…], etc - have been untenable not just since the big negative trials came out this summer, but even by the standards of the early positive trials. Mahmud et al was big and positive and exciting, but it showed that ivermectin patients recovered in about 7 days on average instead of 9. I think the conventional wisdom - that the most extreme ivermectin supporters were mostly gullible rubes who were bamboozled by pseudoscience - was basically accurate.
More tilting at strawmen, sadly. These statements are completely unfounded and say more about Scott than they say about the people he’s targeting. Now of course, the “most extreme” anything “supporters” are mostly “gullible rubes who were bamboozled by pseudoscience,” but that observation carries no evidentiary value. Scott is unlikely to be making the trivial form of this point, so I take it he is talking about fairly middle-of-the-road ivermectin supporters here. For all the errors we’ve been willing to overlook in the course of this deep-dive, Scott seems to subscribe to a zero-tolerance policy.
Mainstream medicine has reacted with slogans like “believe Science”. I don’t know if those kinds of slogans ever help, but they’re especially unhelpful here. A quick look at ivermectin supporters shows their problem is they believed Science too much.
@jonno_bosch I work in hospitality so I need things to return to normal ASAP. I am using Ivermectin as a prophylactic. Hugely influenced by Carvallo trail and Chala trail which showed huge protection@mtskullcrusher @HereComeTheJud @therealjosexy @joeycadre @PeegeRiley @dcwickedestcity @blaireerskine Read Raad. Or Mahmud. Or ICON study from Florida. Or Mexico City hospitalizations study. Or Niaee. Or... Or just type "ivermectin covid" in Google Scholar and read.Well...now, all of a sudden...Ivermectin is cool to take for COVID (as long as it is taken with Doxycycline)? 🧐 But...I thought Ivermectin was "horse dewormer"...smhThey have a very reasonable-sounding belief, which is that if dozens of studies all say a drug works really well, then it probably works really well.
Say it ain’t so!
When they see dozens of studies saying a drug works really well, and the elites saying “no don’t take it!”, their extremely natural conclusion is that it works really well but the elites are covering it up.
Scott’s analysis so far has brought us no reason to disbelieve this extremely natural conclusion.
Sometimes these people even have a specific theory for why elites are covering up ivermectin, like that pharma companies want you to use more expensive patented drugs instead. This theory is extremely plausible. Pharma companies are always trying to convince people to use expensive patented drugs instead of equally good generic alternatives.
This hypothesis is, in fact, so extremely plausible, that Scott didn’t even care to check with the proponents of this extremely plausible hypothesis to see what they were on about. After talking to Gideon Meyerowitz-Katz and other people who presumably watched “Thank You For Smoking” as a manual for life, he concluded “I’ve seen enough!” and got down to business, typing frantically.
Ivermectin believers probably heard about this from the many, many good articles by responsible news outlets, discussing the many, many times pharma companies have tried to trick people into using more expensive patented medications.
I wonder if Scott is talking about the same responsible news outlets that propagated the false—but viral—stories about ivermectin poisoning victims overcrowding hospitals?
Like this ACSH article about Nexium. Or my article on esketamine. Given that dozens of studies said a drug worked, and elites continued to deny it worked, and there are well-known times where elites lie about drugs in order to make money, it was an incredibly reasonable inference that this was one of those times.
But, hear me out on this, have you heard about the worms? They can explain away literally everything! Especially if you put them through a funnel.
So what’s going on here? If Scott is saying that the position he is arguing against is extremely natural, extremely plausible, and incredibly reasonable (yay rule of three) why are we catching him not having investigated even the basic case in favor of that position? Why has he engaged zero of its advocates before coming out with this magnum opus?
Much like the good things Scott said about ivmmeta early on in this essay, he probably didn’t really mean them. I remember someone describing this as the pander/smash technique. First, you say good things about your opponents. This helps set them at ease, but also tells those on your side that you’re not being unreasonable. Then, having gotten everyone to put their defenses down, you spring your rhetorical trap. The (rhetorical) blow hits much harder when it’s not expected.
Pay No Attention To The Man Behind The Curtain
Here is where Scott—triumphant—tells us his real secrets:
If you have a lot of experience with pharma, you know who lies and who doesn’t, and you know what lies they’re willing to tell and which ones they shrink back from.
This is where his whole essay pivots.
Take a minute to read this quote one more time, because the damning admission might have slipped your attention while speed-reading.
Scott tells us that the way to figure out complex questions relating to efficacy of pharmaceuticals is not by reading studies, doing meta-analysis, finding subgroup differences, or doing funnel plots. That would be ridiculous. The way to do it is to have a lot of experience with pharma, figure out who are the truth-tellers, and go with their opinion.
Let’s take this half-seriously for a second. Imagine these reliable truth-tellers exist, and Scott is just being mean by not telling us who they are. How would those reliable truth-tellers have come to hold those reliably true beliefs? Is there some secret conference where actual scientific evidence is presented? Would those reliable truth-tellers have advised in favor or against John Lykoudis’ treatment for h. pylori at the time?
As far as I know, no reputable scientist has ever come out and said ‘esketamine definitely works better than regular ketamine’. The regulatory system just heavily implied it.
I’m sure Scott won’t accept this as evidence, but I did a bit of Googling around on this claim. The first name-brand article in my search results was an article by Psychology Today containing the following quote (emphasis mine):
1. Molecular differences: Understand that Ketamine and Esketamine are similar but they’re not the same. Simply stated, they have the same molecular makeup but Esketamine is more potent than Ketamine. Both can be used to treat resistant depression. For a scientific understanding read more here.
The joke being that when you click the link in search of “a scientific understanding,” you get to a meta-analysis that right at the top says:
Racemic ketamine appears to be more efficacious than esketamine for the treatment of depression. Head to head comparisons are needed to confirm the present findings.
That.. doesn’t seem to support what the paragraph in Psychology Today said. Who wrote it anyway?
Oh, it’s only a.. New York Times and Scientific American contributor? I’m sure she doesn’t count as a reputable scientist for some reason or other that my six minute Googling can’t quite unearth.
But let’s consider what kind of gambit Scott is trying to pull off here. He refers to a case he himself documented, where our system of warped incentives led to a worse treatment being approved for people suffering from treatment-resistant depression (esketamine) than the one most doctors are sure works pretty well but is not specifically approved for this purpose by the FDA (IV ketamine). Somehow, the fact that the unidentified “reputable scientists” did not quite have to come out and straight-up lie to get that to happen is supposed to tell us that they don’t generally straight-up lie if they are put in a corner with their careers hanging in the balance.
That’s like saying that because the world kickboxing champion did not have to exert himself to beat someone up who tried to mug him while he was walking his dog, is evidence in favor of the hypothesis that he cannot exert himself, even when he has to.
I claim that with ivermectin, even the people who don’t usually lie were saying it was ineffective, and they were saying it more directly and decisively than liars usually do.
You know what, at this point these claims of mysterious people who “don’t usually lie” are becoming a nuisance. For one, not lying is not synonymous to always telling the truth. Sometimes people get it wrong. Other times they say nothing. For another, name those names so we can figure out if indeed their track record is as excellent as implied. Because given the text that came before this claim, the probability that it is not supported by evidence is statistically significant.
But most people can’t translate Pharma → English fluently enough to know where the space of “things people routinely lie about and nobody worries about it too much” ends.
I think it’s actually starting to click. It’s almost as if Scott thinks that everything that can be known is actually known, and the key is to find out who knows it. It sounds like he hasn’t considered that the “people who don’t usually lie” may be confused, misled, bamboozled, or just lazily repeating conventional wisdom.
At this point, Scott is not articulating how science works, like he promised at the top of the essay. He is articulating the negation of science, the very system of influence the scientific method was supposed to fight against.
I mean, if his “Pharma → English” translator works as well as it seemed to have worked in this article, I fear I’ll have to report it as defective medical device to the FDA.
I’ve long thought that the inability to sit with uncertainty, the urge to resolve any question immediately is a civilization-level problem, and I am seeing it play out here in real-time.
So they incredibly reasonably assume anything could be a lie. And if you don’t know which statements about pharmaceuticals are lies, “the one that has dozens of studies contradicting it” is a pretty good heuristic!
But if you do know, fuck the studies. Lol.
If you tell these people to “believe Science”, you will just worsen the problem where they trust dozens of scientific studies done by scientists using the scientific method over the pronouncements of the CDC or whoever.
So “believe experts”? That would have been better advice in this case. But the experts have beclowned themselves again and again throughout this pandemic, from the first stirrings of “anyone who worries about coronavirus reaching the US is dog-whistling anti-Chinese racism”, to the Surgeon-General tweeting “Don’t wear a face mask”, to government campaigns focusing entirely on hand-washing (HEPA filters? What are those?) Not only would a recommendation to trust experts be misleading, I don’t even think you could make it work. People would notice how often the experts were wrong, and your public awareness campaign would come to naught.
Oh, I know. Claim that there are some unidentified reliable truth-tellers who know “what’s up,” but only a few chosen clued-in people are able to decode the message embedded in the rate of blinking they do at press conferences. The rest of us should just listen to what they have to say. Let’s run a thought experiment here and ask ourselves how well this strategy would work if we were in the Soviet Union? Or in Mao’s China? Or in one of the unenlightened hellholes that have approved ivermectin as a treatment for COVID-19?
But also: one of the data detectives who exposed some fraudulent ivermectin papers was a medical student, which puts him somewhere between pond scum and hookworms on the Medical Establishment Totem Pole.
Actually he was presenting himself as a “journalist and disinformation researcher” at the time.
These days he says he’s pursuing a “master’s degree in biomedical science”.
Neither of these have anything to do with being a medical student, but Scott can be excused here because the stories the media writes about Lawrence keep repeating this as truth, and nobody seems interested in correcting them.
In any case, who am I to get in the way of a good story, right?
Pierre’s Jihad
Some of the people whose studies he helped sink were distinguished Professors of Medicine and heads of Health Institutes. If anyone interprets “trust experts” as “mere medical students must not publicly challenge heads of Health Institutes”, then we’ve accidentally thrown the fundamental principle of science out with the bathwater.
But Pierre Kory, spiritual leader of the Ivermectin Jihad, is a distinguished critical care doctor.
You mean the guy who testified in the US Senate about how he and his group advocated for the administration of corticosteroids, while the WHO explicitly recommended against? Like, officially:
Is that the same Pierre Kory that only ever prescribed ivermectin as part of a multi-drug protocol? The same spiritual leader that included fluvoxamine in that protocol since early 2021? Even if you believed that ivermectin does nothing, but corticosteroids and fluvoxamine work—like Scott does—you’d want your parents to have Pierre Kory as their doctor if they got COVID-19. You’d recognize that his patients got the kind of treatments that did work, months or even years before everyone else did. And it’s not like they could afford to wait for science to settle.
Meditate on that thought for a minute.
What heuristic tells us “Medical students should be allowed to publicly challenge heads of Health Institutes” but not “Distinguished critical care doctors should be allowed to publicly challenge the CDC”?
Whatever it is, kill it with fire.
Then what about “believe statisticians”? I’ve never heard anyone propose this before, but re-centering the mystique of scientific-expertise in study-analyzers and study-aggregators rather than object-level scientists is…one way you could go, I guess. Statisticians admittedly sort of failed us here: the first several meta-analyses said ivermectin worked. But the statistical process - the idea that studies are raw materials, but it takes skill to turn them into the finished good of scientific knowledge - sort of comes out looking good. If we need to summarize our takeaway in a slogan of exactly two words, one of which is “trust”, you could do worse than this one.
Someone needs to brush up on the Tyranny of Metrics, STAT! An old man named Goodhart is sitting in the corner, mumbling something about paper clips.
(am I secretly suggesting that we make rationality higher status? Maybe, although rationalists did no better here during the early phase of “looks promising so far” than anyone else, and it was researchers digging into the nitty-gritty of the data who really solved this.)
And they didn’t do any better in the late stage either. Apparently “they” think “researchers… really solved this.” Scott’s article is a monument to how truly wrong one can get things and not have most people notice—most rationalists, even. The modern rationalist movement used to be about finding truth. Somewhere along the way, it became about getting a high-end finance job so you can redirect resources to buying malaria nets. I’m exaggerating, but not by much.
If we were to start over, I would want to see a rationalist movement that produces epistemic suicide bombers: people equipped with epistemic humility and courage, as well as the many skills needed to make sense of a complex world, who are willing to take real and permanent reputational hits in order to bring truth to the people. Without being at peace with the eventuality of finding an inconvenient truth and dying in disrepute—like John Lykoudis—efforts to find truth will be hobbled by our self-preservation instincts.
Suffice to say the modern rationalist movement has not even begun to consider this type of effective altruism, and likely never will.
Or maybe this is the wrong level on which to think about this. Maybe there isn’t and can’t be a simple heuristic you can teach everyone in school or via a PR campaign which will lead to them having making good health decisions in an adversarial information environment, without having any negative effects anywhere else. But you also don’t want people to make bad health decisions. So what do you do?
Oh god. There’s another part to this isn’t there?
This is a public peer review of Scott Alexander’s essay on ivermectin. You can find an index containing all the articles in this series here.
The deepest lesson of this saga:
"This article is a monument to how truly wrong one can get things and not have most people notice. Most rationalists, even. The modern rationalist movement used to be about finding truth. Somewhere along the way it became about getting a high-end finance job so you can redirect resources to buying malaria nets. I’m exaggerating, but not by much."
My god. It feels like exposing the Wizard of Oz ... You clearly have brought more ammunition to the table than Biden donated to Zelensky but I am flabbergasted about the sheer quantity of weak spots you've found. I wonder if Scott felt the same pressure I see on other substack writers, to publish too frequently.