Is Ivermectin Literature Particularly Fraudulent?
Recently, I’ve been trying to summarize my thoughts on ivermectin. As part of that, I thought it would be of value to address the most common counter-arguments. The first—and the one most often repeated—is that any positive outcomes are simply the result of fraud.
When we look at the c19early homepage, this chart makes it clear that ivermectin has a very strong signal of efficacy. Even looking at individual endpoints—or only RCTs—or even only mortality results from RCTs—the signal is still there. So what gives? How can we explain this? Are the people objecting actually right?
Does Fraud Explain the Observation?
Leaving aside—for the moment—the fact that the chart above has removed studies strongly suspected of fraud, can we try to evaluate the cleanest form of this argument?
Unfortunately, the researchers that have been at the forefront of this argument (Gideon Meyerowitz-Katz, Nick Brown, Jack Lawrence, James Heathers) have never made their data available, despite promising to do so.
Over 560 days ago, I was promised that the data would be made public “soon,” but no data has been forthcoming, despite articles in the BBC, BMJ, Nature, and The Atlantic touting the results of this unpublished work. This despite these same researchers taking a zero-tolerance attitude toward studies that don’t share data upon request:
As such, we will have to attempt to reconstruct the dataset that wasn’t shared. This is what was written in the BBC article:
The group of independent scientists examined virtually every randomised controlled trial (RCT) on ivermectin and Covid - in theory the highest quality evidence - including all the key studies regularly cited by the drug's promoters.
RCTs involve people being randomly chosen to receive either the drug which is being tested or a placebo - a dummy drug with no active properties.
The team also looked at six particularly influential observational trials. This type of trial looks at what happens to people who are taking the drug anyway, so can be biased by the types of people who choose to take the treatment.
Out of a total of 26 studies examined, there was evidence in five that the data may have been faked - for example they contained virtually impossible numbers or rows of identical patients copied and pasted.
In a further five there were major red flags - for example, numbers didn't add up, percentages were calculated incorrectly or local health bodies weren't aware they had taken place.
On top of these flawed trials, there were 14 authors of studies who failed to send data back. The independent scientists have flagged this as a possible indicator of fraud.
What we learn from this is that they looked at “virtually every randomized trial” and six “particularly influential” observational trials—for a total of 26 studies examined— which means they looked at 20 RCTs. Given that as early as August 2021, there had been 30 RCTs on ivermectin, it’s hard to know how “virtually all” maps to the real world, but I suppose we will have to take the BBC’s word for it.
If we are to believe the claims made by the author of the BBC article, there were five where the data may have been faked, a further five where there were “major red flags,” and an additional 14 that “failed to send data back.” This means there were only two studies that remain standing? That can’t be true, given that the authors have praised trials like Mahmud, Zoni/Vallejos, and Chaccour:
Another member of this team tried to address my point in an article in The Atlantic. He made this argument in particular:
Over the past six months, we’ve examined about 30 studies of the drug’s use for treating or preventing COVID-19, focusing on randomized studies, or nonrandomized ones that have been influential, with at least 100 participants. We’ve reached out directly to the authors of these studies to discuss our findings, sometimes engaging in lengthy back-and-forths; when appropriate, we’ve sent messages to the journals in which studies have been published. In our opinion, a bare minimum of five ivermectin papers are either misconceived, inaccurate, or otherwise based on studies that cannot exist as described. One study has already been withdrawn on the basis of our work; the other four very much should be.
[…]
All of the above may not sound that bad. If five out of 30 trials have serious problems, perhaps that means the other 25 are up to snuff. That’s 83 percent! You might be tempted to think of these papers as being like cheaply made light bulbs: Once we’ve discarded the duds with broken filaments, we can just use the “good” ones.
That’s not how any of this works. We can locate obvious errors in a research paper only by reanalyzing the numbers on which the paper is based, so it’s likely that we’ve missed some other, more abstract problems. Also, we have only so much time in the day, and forensic peer review can take weeks or months per paper. We don’t pick papers to examine at random, so it’s possible that the data from the 30 papers we chose are somewhat more reliable, on average, than the rest. A better analogy would be to think of the papers as new cars: If five out of 30 were guaranteed to explode as soon as they entered a freeway on-ramp, you would prefer to take the bus.
Most problematic, the studies we are certain are unreliable happen to be the same ones that show ivermectin as most effective. In general, we’ve found that many of the inconclusive trials appear to have been adequately conducted. Those of reasonable size with spectacular results, implying the miraculous effects that have garnered so much public attention and digital notoriety, have not.
There is so much wrong with the reasoning here that it would require an entire analysis just to untangle it. Thankfully, I have a Twitter thread to do just that, so we can spare this article from that untangling. The key statistic to take away is that there is a claim of 17% studies with “serious problems.” Is that particularly unusual? Not according to the BMJ:
Others have found similar results, and Mol’s best guess is that about 20% of trials are false. Very few of these papers are retracted.
It seems that—even with this hand-picked dataset—the fraud researchers have not been able to make a case that ivermectin research is particularly fraud-infested, despite their comments to the contrary:
Let’s Dig (Even) Deeper
Disclaimer: I’m going to have to make some assumptions about which trials are implicated here—seeing as the authors have not shared their dataset—and what is in the BBC article appears to be unreliable. I have made every effort to be fair, but an easy way to improve my commentary would be to release the data, as promised.
Let’s focus on the five studies with “evidence the data may have been faked.” This would include Elgazzar and Samaha which have been retracted for such issues. But which are the other three? Trawling through tweets, I found this, which may give us some hints:
Two of the Carvallo studies—as well as one of the Cadegianni trials—have been accused of data manipulation by this group, so I will assume they are in the set of “potentially fraudulent” trials. If so, we would be looking at two of 20 RCTs and three of six observational trials being tagged as “potentially fraudulent.” This is confirmed by this tweet, whose traces I found on the internet (Sheldrick’s Twitter account is now private).
As background we have publicly bolt-gunned a number of studies of ivermectin. I don't mean niggled, I mean put down, here's a selection of the most important:
– Elgazzar
– Carvallo
– Cadegiani
– Samaha
– Niaee
2/19— Kyle Sheldrick (@K_Sheldrick) October 10, 2021
Niaee is being accused of randomization failure, so that doesn’t rise to the level of “fraud”, and similarly for Samaha, the accusation as far as I know is not data fabrication, but also this group has not seen the data, to the best of my knowledge.
This lopsidedness may explain the reticence to share raw data, contrary to the group’s stated values. Given that the observational trials are a very small subset that have been selected with the subjective criterion of being “particularly influential,” I won’t spend too much time on that. The more interesting claim is around the RCTs.
The most important question we have to ask ourselves is this: is it particularly shocking to have two of 20 RCTs in a body of literature be potentially fraudulent?
Much of the methodology this team of researchers uses originates from the work of Dr. John Carlisle. John Ioannides summarizes Carlisle’s findings as follows:
Carlisle probed the submitted papers for all 526 trials considered by Anaesthesia over 3 years (February 2017–March 2020) and also obtained individual level data from the authors of 153 trials. Availability of individual level data increased the odds of a trial being called false or zombie by 47-fold and 79-fold, respectively. Among the 153 trials examined in depth, 44% were deemed to be false and 26% were deemed to be zombies. Very few false and zombie trials could be detected based just on the originally submitted manuscript without the benefit of accessing individual level data.
Indeed, both the Elgazzar and Shouman studies have been uncovered on the basis of access to their underlying data.
As such, we can zoom in on a more narrow question: how many ivermectin RCTs shared their data with these researchers?
From this and other tweets, we can put together a list of RCTs that this group almost certainly has seen data for: Biber/Schwarz, Babalola, Chaccour, Mahmud, Lopez-Medina, Mohan, Ravikirti, Vallejos/Zoni, Elgazzar, Beltran-Gonzalez, and Shouman/Raad.
This would mean that—at most—they have identified 18% (2 of 11) of these as potentially fraudulent—or in the terminology of Carlisle—“zombie RCTs.” If they have seen data for more than these trials, then the percentage drops further. Given that Carlisle classified 26% of the trials he investigated with access to raw data as “zombies,” there’s only one conclusion we can come to: ivermectin trials that have been examined by adversarial investigators show a lower-than-expected level of “fraudulence.” Given that the ivermectin literature has been investigated as thoroughly as any other (which is a good thing), we can have some level of comfort that if anything, ivermectin research has been scrutinized at least as well as any other.
I realize that some may take issue with my characterizing this group of researchers as “adversarial,” so before we wrap up, some receipts:
With this, I think my case has been sufficiently made.
In Conclusion
We have taken two approaches to try to quantify the level of “fraud” in the ivermectin literature. One is to take the fraud researchers’ work at face value and compare it with the expected baseline. It appears that by that measure, ivermectin research is below-average in the degree of fraud present (17% vs 20%). The second approach is to try to guess the number of studies these researchers had access to Individual Patient Data (IPD) for, and estimate the degree of “zombie” trials in that set. By that metric also, ivermectin research does not appear out of the ordinary compared to other well-scrutinized literature (18% vs 26%) and if anything, is less suspicious than expected.
Should this make us concerned about medical research in general? Absolutely. Should it make us particularly concerned about the quality of ivermectin literature? Not unless we’re willing to throw out the entirety of modern medicine. Whatever is happening with ivermectin, it doesn’t seem that “fraud” is the explanation.
Fraud is almost always preceded by a motive which, if successful, gives the fraudulent party a benefit; and generally weighed against the risk of being caught.
A positive Ivermectin study provides the fraudulent researcher with:
1. $0 as the drug is generic
2. Character assassination, slander, defunding of research, firing from position.
I am waiting to hear someone provide even the slightest motivation for submitting a fraudulent positive Ivermectin paper - you still get 1 and 2, plus your a fraud.
On the other hand, the vaccine cartel would pay handsomely for both negative and “obviously fraudulent” studies as both are used to discredit the single most efficacious drug for treatment of any ailment, in the history of evidence based medicine.
Centuries of precedent proves motive is fundamental to any accusation of fraud or crime.
And I don’t think “a completely unrelated evil band of 1000’s of Doctors and Researches with impeccable records, hell bent on preventing the benevolent vaccine-cartels from cashing in on society” cuts it.
Almost all of the large-scale ivermectin RCTs were obviously DESIGNED TO FAIL by doing most or all of the following:
1. enrolling low-risk patients who didn’t need ivermectin to begin with
2. delaying the ivermectin until almost a week after symptoms started
3. not giving the ivermectin with food to optimize absorption
4. not combining the ivermectin with other agents that are usually used in conjunction with it (even Paxlovid, which reduced COVID deaths from 13 to 0 in its RCT, is a COMBINATION drug)
5. stopping the ivermectin after just 3 days or so of treatment
After all this, of course those large-scale ($$$) RCTs conveniently found "no significant effect” of ivermectin. But even though many RCTs were designed to fail, the average effect in all published RCTs was that ivermectin prevented about 1 in 3 COVID deaths: https://c19ivm.org/meta.html#fig_fprd.
So actually, yes, the ivermectin literature IS particularly fraudulent: it is full of RCTs that were deliberately set up to generate negative results.