This is a public peer review of Scott Alexander’s essay on ivermectin. You can find an index containing all the articles in this series here.
We’ve discussed how Scott stealth-iterated on the hypothesis which was the focus of his ivermectin article. We’ve separately discussed the evidence for the hypothesis itself.
For this installment, we’ll break down the rhetorical portion of Scott’s piece, see how it works, and what we can learn from it. I’ve long felt that Scott’s world-class writing ability gets in the way of representing the truth in this case—and perhaps in others. His writing entertains people who would otherwise be able to highlight the many logical errors. It also allows Scott himself to consider the strongly positive reception of his piece as evidence that the logic is sound.
As we have seen repeatedly in this series, sound logic could not be further from the truth. But to see the flaws, we’ve had to slow the piece down and consider each sentence separately, lest we get carried away by the spectacular stream of prose.
Today, I’ll do some rhetorical analysis on the tip of the spear of the essay— the part that delivers the infectious memetic payload—worms!
The Synthesis
Hopefully you learned something interesting about yourself there. But my answer is: worms!
As several doctors and researchers have pointed out (h/t especially Avi Bitterman and David Boulware), the most impressive studies come from places that are teeming with worms.
By “teeming with worms,” Scott means that there’s more than 8.1% prevalence, according to model-adjusted results of one study. Figuring out the prevalence of Strongyloides infection (the only relevant kind) is an open problem. So to say that half the people in the world are "teeming with worms” is bordering on false stereotyping.
Mahmud from Bangladesh, Ravakirti from East India, Lopez-Medina from Colombia, etc.
Except, of course, there’s nothing particularly “impressive” about Ravikirti, previously noted by Scott. If we’re talking about impressive results, Lopez-Medina showed a very weak signal of effectiveness. However, these three studies are in fact the three studies the meta-analysis Scott is about to cite uses for the high-prevalence areas, alongside one more study that hasn’t yet been covered in his article. So when he casually brings up these studies as “most impressive,” and completes the statement with “etc.,” there’s nothing he’s talked about thus far that could actually fit into that “etc.” bucket. Good rhetoric, but not exactly the hallmark of honest exposition.
Here’s the prevalence of roundworm infections by country (source).
It should be clarified that this map is about a different species of worm (Ascaris lumbricoides), and not the one pinpointed by Dr. Bitterman and Dr. Boulware (Strongyloides stercoralis). Not sure why Scott chose this one, but without that clarification, you may get the idea that, for instance, the relevant worm species is not present in the US, which isn’t accurate.
Here’s a map on Strongyloides from the main reference Dr. Avi Bitterman used:
But alongside roundworms, there are threadworms, hookworms, blood flukes, liver flukes, nematodes, trematodes, all sorts of worms. Add them all up and somewhere between half and a quarter of people in the developing world have at least one parasitic worm in their body.
Keep in mind that this point is entirely irrelevant to the hypothesis—only Strongyloides stercoralis is implicated in the hypothesis Scott is developing. Listing worms is highly entertaining, however.
Being full of worms may impact your ability to fight coronavirus.
Note again the false picture being created by saying people are “full of worms,” as if they’re about to burst open, unleashing millions of larvae on the world at any moment.
Gluchowska et al write:
Helminth [ie worm] infections are among the most common infectious diseases. Bradbury et al. highlight the possible negative interactions between helminth infection and COVID-19 severity in helminth-endemic regions and note that alterations in the gut microbiome associated with helminth infection appear to have systemic immunomodulatory effects. It has also been proposed that helminth co-infection may increase the morbidity and mortality of COVID-19, because the immune system cannot efficiently respond to the virus; in addition, vaccines will be less effective for these patients, but treatment and prevention of helminth infections might reduce the negative effect of COVID-19. During millennia of parasite-host coevolution helminths evolved mechanisms suppressing the host immune responses, which may mitigate vaccine efficacy and increase severity of other infectious diseases.
What if we read more of the same paper Scott cites? You see, the Gluchowska et al. paper is very neutrally named, “The New Status of Parasitic Diseases in the COVID-19 Pandemic—Risk Factors or Protective Agents?” It also contains the opposing hypothesis:
6. Can Parasites Protect Us from the COVID-19?
The low incidence rates of COVID-19 in Africa [74,77,78,79] are of high interest to scientists and WHO authorities, and it has been hypothesized that this could be a result of the increased exposure to parasites in less developed countries: the populations of Africa and Latin America are much more likely to suffer from parasitic diseases than those of more highly developed countries.
Helminth infection entails various forms of immunomodulation, resulting in an increased susceptibility to some infections, a decreased susceptibility to others, and changes in the intensity of allergic, autoimmune and inflammatory diseases; it has also been proposed that infection may account for inadequate responses to vaccines and, possibly, better tolerance of SARS-CoV-2 infection [5,80,81].
So here we have Scott citing a paper that is ambivalent about what effect, if any, helminths have on COVID-19 infection—as if it were strongly supportive of his position. Not good practice, to say the least.
Treatment of worm infections might reduce the negative effect of COVID-19! And ivermectin is a deworming drug! You can see where this is going…
Yep.
The most relevant species of worm here is the roundworm Strongyloides stercoralis. Among the commonest treatments for COVID-19 is corticosteroids, a type of immunosuppresant drug. The types of immune responses it suppresses do more harm than good in coronavirus, so turning them off limits collateral damage and makes patients better on net. But these are also the types of immune responses that control Strongyloides. If you turn them off even very briefly, the worms multiply out of control, you get what’s called “Strongyloides hyperinfection”, and pretty often you die.
Scott presents this as a certainty (“If you turn them off even very briefly, the worms multiply out of control…”) but in the medical literature, it’s extremely unclear how often this happens, what dose and duration, or other factors must be present to trigger it.
As we discussed in a previous article, Strongyloides hyperinfection is a relatively rare occurrence. A paper that searched all the literature between 1970 and 2010, came up with 133 cases described, or about 3.3 cases per year. Naturally, not every case is written up, but even if we assume that only 1 in 1,000 cases is, we’re still talking about 3,000 cases a year. The paper from which the maps above were taken, claims Strongyloides currently infects 614 million people. This would mean (back of the envelope) something like a 1/200,000 occurrence of hyperinfection—the kind of thing we can safely call incredibly rare—even though steroids are administered frequently. Also, only a portion of those hyperinfection events were lethal.
Scott gives readers the wrong impression here, on an assertion that is critical to the hypothesis he is presenting.
According to the WHO:
If you think Scott is quoting an official WHO recommendation, guess again. He is quoting from a news article posted on the WHO website, under “departmental news.”
But let’s keep reading:
The current COVID-19 pandemic serves to highlight the risk of using systemic corticosteroids and, to a lesser extent, other immunosuppressive therapy, in populations with significant risk of underlying strongyloidiasis. Cases of strongyloidiasis hyperinfection in the setting of corticosteroid use as COVID-19 therapy have been described and draw attention to the necessity of addressing the risk of iatrogenic strongyloidiasis hyperinfection syndrome in infected individuals prior to corticosteroid administration.
Although this has gained importance in the midst of a pandemic where corticosteroids are one of few therapies shown to improve mortality, its relevance is much broader given that corticosteroids and other immunosuppressive therapies have become increasingly common in treatment of chronic diseases (e.g. asthma or certain rheumatologic conditions).
So you need to “address the risk” of strongyloides infection during COVID treatment in roundworm-endemic areas. And how might you address this, WHO?
Treatment of chronic strongyloidiasis with ivermectin 200 µg/kg per day orally x 1-2 days is considered safe with potential contraindications including possible Loa loa infection (endemic in West and Central Africa), pregnancy, and weight <15kg.
Given ivermectin’s safety profile, the United States has utilized presumptive treatment with ivermectin for strongyloidiasis in refugees resettling from endemic areas, and both Canada and the European Centre for Disease Prevention and Control have issued guidance on presumptive treatment to avoid hyperinfection in at risk populations. Screening and treatment, or where not available, addition of ivermectin to mass drug administration programs should be studied and considered.
This is serious and common enough that, if you’re not going to screen for it, it might be worth “add[ing] ivermectin to mass drug administration programs” in affected areas!
Scott is inserting a premise: that it is something that is “common enough.” This is not borne out by the data. Also, the WHO has not added anything about ivermectin to its proposed protocol for administration of corticosteroids. All this one departmental news article on the WHO website says is that it “should be considered.” Well, if they did consider it, they don’t seem to have moved it forward with it.
And, to be clear, Scott’s implication is that actual doctors practicing actual medicine in actual affected areas have not thought of this, but some Information Officer at the WHO somehow knows more than the people on the front lines—who are somehow losing a ton of patients to hyperinfection since the introduction of steroids as standard of care—but haven’t noticed, or thought to modify their local protocols to address this.
In fact, if you want to see what an actual WHO recommendation looks like, this is more like it.
The real answer to all this was written by Scott himself. It was called “beware the man of one study.” The hypothesis he raises is interesting and worth studying, but we are nowhere near far enough into the process to elevate it to any sort of definitive explanation for anything. In fact, what Scott does here is the antithesis of the science he nominally is in favor of: he plucks a specific hypothesis from obscurity, overstates several lines of evidence, and positions it as the most prevalent explanation for studies that have orders of magnitude more statistical power than the hypothesis itself. And that’s before we investigate the paper, and realize that when we adjust the prevalence rates to harmonize them across data sources, the effect seems to go away.
Does This Analysis Pass Scott’s Tests for Good Science?
Now, remember, Scott promised his readers no less than an explanation of how science works:
What a great opportunity to exercise our study-analyzing muscles! To learn stuff about how science works which we can then apply to less well-traveled terrain! Sure, you read the articles saying that experts had concluded the studies were wrong. But did you really develop a gears-level understanding of what was going on? That’s what we have a chance to get here!
And throughout his essay, he wielded an implicit list of red (or at least deep orange) lines that he would throw out studies for, especially if 3 or more were crossed:
Not explicitly declaring a primary outcome in the text of the paper
Not being an RCT
Pre-registration issues
Not published and peer-reviewed in a high-impact journal
Not having a plausible result
Being caught in Carlisle-style filtering (or looking like you might be)
The authors’ reputation in relation to other studies they’ve published
Gideon Meyerowitz-Katz not being a fan
And somehow, he ended up leading his readers to put their faith in a study that was:
Correlational (which he has previously dumped on)
Not pre-registered (so no problems with protocol or primary endpoint)
Using iterated analyses he called others out for (wrongly, even)
No peer-reviewed and published paper available that could be subjected to critique
Many of his other criteria—RCT, primary outcome, Carlisle-style filtering, prior reputation—simply didn’t apply here. The ones that did, didn’t look confirmatory.
To be clear, I don’t mean that his concerns on other papers were necessarily valid, only that if he’s going to use that criteria, he must apply them consistently—especially if he is going to represent what he is doing as not the workings of his own mind, but science!
What’s more, even when digging into the Strongyloides hypothesis, I felt somewhat uneasy, as it could well be that such a correlation could emerge about any generic drug—with one known application—being tested for another purpose. How could I guarantee I was not fooling myself by privileging the hypothesis? After all, 12 studies can be separated into two groups in a few thousand ways; some of those will show significant group differences. If a hypothesis can be tailored to one of those groupings, should we throw away the whole set of studies?
Besides, I don’t think Scott tested fluvoxamine for its effects on depression before pushing for it to be approved. Yes, depression is a risk factor for COVID-19. Also, fluvoxamine studies seem to be—after a quick look—mostly of benefit to women, who are also most likely to suffer from depression.
Come to think of it, it may be worth digging into that hypothesis, if only to set a floor for considering such a correlation at all. If the association of fluvoxamine with depression, and depression with COVID-19 outcomes is stronger than ivermectin with worms, and worms with COVID-19, one must either give up their support for the method itself, or at the very least throw out support for fluvoxamine as well as any other drug for which such an objection can be credibly raised (which, I suspect, is a non-trivial subset).
The Perfect Meme
As my friend @gfodor said on Twitter:
Sadly, Scott chose to apply his substantial rhetorical skills to launch this particular hypothesis into the consciousness of the world unopposed; what it lacks in smoking-gun evidence it makes up for in memetic fitness. It gives readers the sense that they’re smart, that they’ve done their homework, and discovered the deep and intriguing source of the outrage. Everyone was kinda right after all, right? The only reasonable move now is to stand down, accept the results of Science Done Right, and move on with their lives.
Except, of course, none of it is true. Scott had to violate his own principles, and rob a couple of graves to get his essay to sound convincing. There’s cutting corners, and then there’s lighting libraries worth of evidence on fire and then somehow convincing people that there’s no such thing as a library and there never was.
This wasn’t a case of mere rhetorical skill used to convince. It was a case of rhetorical skill serving to obscure glaring flaws in an underlying logical structure that people needed to be entertained away from spotting. That entertainment often came in the form of cheap shots aimed at working researchers who did nothing wrong but produce results someone didn’t like. And, please note, I’m not saying it was Scott’s intent to do this. I’m saying that this was the effect of the essay as delivered.
And deliver it did. If you’re not following me on Twitter (heresy!) here are a few of the many places the meme went:
Next up, we’ll be talking funnel plots and viral load studies, so get excited and hang on tight!
This is a public peer review of Scott Alexander’s essay on ivermectin. You can find an index containing all the articles in this series here.
This has been a fascinating series. Well done.
I think you've done a good job dotting i's and crossing t's; but, at a more simplistic and intuitive level, the thing that stands out for me is the caption, "A defiant Flavio Cadegiani. Imagine a guy who looks like this telling you to take ultra-high-dose antiandrogens."
It's so shockingly juvenile. And a microcosm for the point you make that Scott used "rhetorical skill serving to obscure glaring flaws in an underlying logical structure that people needed to be entertained away from spotting".
What shocks me most is how I just skimmed over that caption (and many other points you've raised) the first time I read it and how I was duped. Very humbling.
To be fair, glib arguments that don't hold water are The Economist's bread and butter.