Also, I wanted to let you know the link you have for part two says it is private, so I cannot view the article. I would like to share this series in response to that commenter but want to get the correct link first. Thanks, Alexandros!
I generally don't like talking about motives either; but I agree calling out unitaid is the right thing to do. Likely the deafening silence will continue, and there is little legal basis for taking action against them, if any. But if I was science czar id seek to have their organization permanently blacklisted from my country. Not out of concern for IVM; but out of a general concern for scientific ethics. People talk a lot about the potential for money to have implicit influences on science; but I do not recall ever hearing of something so brazen as this affair involving unitaid. The way things are going, people are going to start thinking that you disclose your funding sources not for an implicit risk of bias, but that funding sources literally dictating conclusions is infact 'the new normal'.
Haven't read this one yet, but Tess Lawrie and her group, with a lot of experience in doing Cochrane meta-analyses under their belt, had a pretty devastating response to a prior iteration of the analysis. Unless the issues are resolved in the new version, I expect the analysis still carries very little weight.
I guess the dilemma is that the opinion of Popp et al aligns with that of Scott Alexander, despite your detailed expose of the inadequacies of Scott’s analysis.
In general terms, the established medical community do not care about Scott Alexander’s opinion (nor for that matter yours). However, major regulatory groups such as the FDA, TGA etc do care what Cochrane reviews say.
In an ideal world, and this may sound crazy, what I would like to see happen is this: Scott Alexander recognises the points you make and works together with you to actually summarise all includable data then perform a robust meta-analysis on that data. Then compare that with the inclusions/exclusions of the recent Cochrane review and if you feel that the Cochrane review is inadequate, take the appropriate steps to challenge their results.
Sadly, I feel none of that will happen.
The Oxford trial results will come out soon (with predictable outcomes), reinforcing the Cochrane/Alexander position. Many doctors however will ignore all this and continue using ivermectin with surprisingly good results!
Tess Lawrie is the antidote to Popp. She's done the work, and the way Cochrane wasted her time and got rid of her when she refused to work with Cochrane says more about them than the Popp review ever could.
What I can do is respond to Scott, because I speak the same language, and I can dig in and find issues.
Of course, the establishment will always have the first word in the moment, but history has a more dispassionate view on such matters, and its important to put this information on the record.
Indeed it would be great if all issues were discussed to their eventual conclusion; but such isnt the world we live in. Id love to see a live debate, between, say, the authors of this Cochrane review, Tess Lawrie, and say, a spokeperson for ivmmeta. But I doubt it will happen.
Even more so than for non-meta studies, which you can design for negative results, with meta studies, its ridiculously easy to design the inclusion criteria to tailor to your desired outcome. The description for inclusion criteria are made to sound innocous; but it happens more often than not, that different meta analyses end up with barely overlapping sets of studies; for reasons that you will never quite be able to dissect retroactively.
One detail that stood out to me here is the following 'Co‐interventions had to be the same in both study arms'. Sounds reasonable. But many if not most IVM studies were part of some cocktail, including antibiotics for instance. True; to be strict the study only proves something about the cocktail. But realistically... do we really believe the antibiotic cured a viral disease, or are you just fishing for reasons to exclude the studies you dont like? Because there is an infinite such 'reasonable restrictions' you can pick from, to arrive at almost any subset of studies of your choosing. Cochrane sure arrived at a small subset.
I think ivmmeta is leading the way here in terms of making the data transparent. All ifs and buts of all papers are openly discussed; not behind a paywall, or anonimity of peer review. And they make it easy to see the impact of running the meta analysis in different ways, with various inclusion criteria.
What exactly these Cochrane reviewers were up to, and if it makes any sense, would take weeks to get to the bottom of even for someone like Lawrie; and months for a dedicated observer not intimately familiar with the whole literature.
Ivmmeta/c19early is an incredible resource, possibly the best I’ve ever seen. Frankly puts Cochrane to shame.
I fear almost nothing will ever get those who have already made up their mind (FDA/WHO/TGA etc) to reverse their opinion and acknowledge that IVM may have a role in Covid.
I find it amusing that those who cite Cochrane as the be all and end all regarding IVM in Covid still use the pulse oximeter, despite Cochrane clearly identifying that there is no evidence that it approves outcomes. The justification being that clearly the pulse oximeter is a good thing just the evidence can’t prove it...would be laughable hypocrisy if it wasn’t so depressing
Cochrane review actually seems good quality imo, and seems on par with their other reviews. But it reflects the difficulty of synthesising ivm evidence. E.g. there is substantial uncertainty in mortality outcome (not unexpected for outpatient data); they also weren’t able to analyse hospitalisation. Many more. I don't think "concludes ivermectin doesn't work" is accurate; Rather it shows a lack of synthesis-able evidence. For the first review version, it'd be even more silly to say "ivm doesn't work" as they barely included any trials in that one (literally 1-2 studies for outcomes, with CIs that could fit a cruise ship). Authors said they were very uncertain and asked for more studies. Should not be confused with/interpreted as evidence of absence/ineffectiveness!
Scott Alexander (as he prefers to be called in his internet writing) consistently refers to "insignificant" results. In reading hundreds of scientific papers, I have never seen that word used to indicate that the data cannot reject the null hypothesis of no effect. The correct and universally used term is "non-significant." I consider SA's changing the terminology as another rhetorical trick intended to dismiss the trial even if it shows indications of an effect, as for example a non-significant dose response effect.
Non-significant is already a stretch of the verbal toolkit which makes me extremely uncomfortable. To further modify those linguistic magic tricks is unacceptable to anyone who understands even the beginnings of the underlying debate.
As someone who began reading the scientific literature at age 22, I accepted "non-significant" as neutral as to whether further testing for an effect would be called for. Whether it is neutral though seems case specific. In my example of a non-significant dose response effect, perhaps non-significant is too loaded against the hypothesis. The really correct way to communicate non-significant results would be through a rather long sentence referring to the power of the test, the proximity of the p-value to the criterion of significance, and to other aspects of experimental design.
That detail does not seem to carry any semantic weight to me? Both insignificant and non-significant are just negations of significant to me?
Im much more bothered by the repeated conflating of absence of evidence, and evidence of absence. Now in scotts defense he hardly invented this confusion and its near-universal in science journalism. But I was expecting better of him on that front, even if he is not a domain expert and he might get a lot of the details wrong.
I just wish he, or someone else, woild look at Ivermectin use worldwide and the scientific versus political merits. There is so much media obfuscation that prevents access to reliable data.
"Well, this took a strange turn towards the conspiratorial"
This whole pandemic has been full of scientists, public health agencies, and the media deliberately and consciously misleading the public. It's not that strange.
THANK YOU for tackling Scott's smear piece on ivermectin, Alexandros. I only learned about it myself when a commenter at The Burning Platform version (https://www.theburningplatform.com/2022/07/24/letter-to-alex-berenson-on-world-ivermectin-day/comment-page-1/) of my "Letter to Alex Berenson on World Ivermectin Day" (https://margaretannaalice.substack.com/p/letter-to-alex-berenson-on-world) shared a link to Scott's article, stating, "Deep research assessment by Scott Alexander which suggests the author of this piece has her head firmly up her ass. Her credibility just dropped to nil."
Also, I wanted to let you know the link you have for part two says it is private, so I cannot view the article. I would like to share this series in response to that commenter but want to get the correct link first. Thanks, Alexandros!
Thank you. Should be fixed now.
There are also two previous pieces I will eventually incorporate:
1. https://doyourownresearch.substack.com/p/a-conflict-of-blurred-visions
2. https://doyourownresearch.substack.com/p/scott-alexander-corrects-error-ivermectin
Arguably you should read those first before diving in with these more detailed posts.
It’s working now—thank you! And I appreciate the additional links.
I generally don't like talking about motives either; but I agree calling out unitaid is the right thing to do. Likely the deafening silence will continue, and there is little legal basis for taking action against them, if any. But if I was science czar id seek to have their organization permanently blacklisted from my country. Not out of concern for IVM; but out of a general concern for scientific ethics. People talk a lot about the potential for money to have implicit influences on science; but I do not recall ever hearing of something so brazen as this affair involving unitaid. The way things are going, people are going to start thinking that you disclose your funding sources not for an implicit risk of bias, but that funding sources literally dictating conclusions is infact 'the new normal'.
Hi Alexandros, great article yet again.
What do you make of the recent Cochrane review that concludes the same as the first ie Ivermectin doesn’t work against Covid?
https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD015017.pub3/full
Haven't read this one yet, but Tess Lawrie and her group, with a lot of experience in doing Cochrane meta-analyses under their belt, had a pretty devastating response to a prior iteration of the analysis. Unless the issues are resolved in the new version, I expect the analysis still carries very little weight.
https://bird-group.org/rapid-response-to-editor-of-bmj-evidence-based-medicine-re-popp-et-al/
I guess the dilemma is that the opinion of Popp et al aligns with that of Scott Alexander, despite your detailed expose of the inadequacies of Scott’s analysis.
In general terms, the established medical community do not care about Scott Alexander’s opinion (nor for that matter yours). However, major regulatory groups such as the FDA, TGA etc do care what Cochrane reviews say.
In an ideal world, and this may sound crazy, what I would like to see happen is this: Scott Alexander recognises the points you make and works together with you to actually summarise all includable data then perform a robust meta-analysis on that data. Then compare that with the inclusions/exclusions of the recent Cochrane review and if you feel that the Cochrane review is inadequate, take the appropriate steps to challenge their results.
Sadly, I feel none of that will happen.
The Oxford trial results will come out soon (with predictable outcomes), reinforcing the Cochrane/Alexander position. Many doctors however will ignore all this and continue using ivermectin with surprisingly good results!
Tess Lawrie is the antidote to Popp. She's done the work, and the way Cochrane wasted her time and got rid of her when she refused to work with Cochrane says more about them than the Popp review ever could.
What I can do is respond to Scott, because I speak the same language, and I can dig in and find issues.
Of course, the establishment will always have the first word in the moment, but history has a more dispassionate view on such matters, and its important to put this information on the record.
Indeed it would be great if all issues were discussed to their eventual conclusion; but such isnt the world we live in. Id love to see a live debate, between, say, the authors of this Cochrane review, Tess Lawrie, and say, a spokeperson for ivmmeta. But I doubt it will happen.
Even more so than for non-meta studies, which you can design for negative results, with meta studies, its ridiculously easy to design the inclusion criteria to tailor to your desired outcome. The description for inclusion criteria are made to sound innocous; but it happens more often than not, that different meta analyses end up with barely overlapping sets of studies; for reasons that you will never quite be able to dissect retroactively.
One detail that stood out to me here is the following 'Co‐interventions had to be the same in both study arms'. Sounds reasonable. But many if not most IVM studies were part of some cocktail, including antibiotics for instance. True; to be strict the study only proves something about the cocktail. But realistically... do we really believe the antibiotic cured a viral disease, or are you just fishing for reasons to exclude the studies you dont like? Because there is an infinite such 'reasonable restrictions' you can pick from, to arrive at almost any subset of studies of your choosing. Cochrane sure arrived at a small subset.
I think ivmmeta is leading the way here in terms of making the data transparent. All ifs and buts of all papers are openly discussed; not behind a paywall, or anonimity of peer review. And they make it easy to see the impact of running the meta analysis in different ways, with various inclusion criteria.
What exactly these Cochrane reviewers were up to, and if it makes any sense, would take weeks to get to the bottom of even for someone like Lawrie; and months for a dedicated observer not intimately familiar with the whole literature.
Ivmmeta/c19early is an incredible resource, possibly the best I’ve ever seen. Frankly puts Cochrane to shame.
I fear almost nothing will ever get those who have already made up their mind (FDA/WHO/TGA etc) to reverse their opinion and acknowledge that IVM may have a role in Covid.
I find it amusing that those who cite Cochrane as the be all and end all regarding IVM in Covid still use the pulse oximeter, despite Cochrane clearly identifying that there is no evidence that it approves outcomes. The justification being that clearly the pulse oximeter is a good thing just the evidence can’t prove it...would be laughable hypocrisy if it wasn’t so depressing
Cochrane review actually seems good quality imo, and seems on par with their other reviews. But it reflects the difficulty of synthesising ivm evidence. E.g. there is substantial uncertainty in mortality outcome (not unexpected for outpatient data); they also weren’t able to analyse hospitalisation. Many more. I don't think "concludes ivermectin doesn't work" is accurate; Rather it shows a lack of synthesis-able evidence. For the first review version, it'd be even more silly to say "ivm doesn't work" as they barely included any trials in that one (literally 1-2 studies for outcomes, with CIs that could fit a cruise ship). Authors said they were very uncertain and asked for more studies. Should not be confused with/interpreted as evidence of absence/ineffectiveness!
Scott Alexander (as he prefers to be called in his internet writing) consistently refers to "insignificant" results. In reading hundreds of scientific papers, I have never seen that word used to indicate that the data cannot reject the null hypothesis of no effect. The correct and universally used term is "non-significant." I consider SA's changing the terminology as another rhetorical trick intended to dismiss the trial even if it shows indications of an effect, as for example a non-significant dose response effect.
Non-significant is already a stretch of the verbal toolkit which makes me extremely uncomfortable. To further modify those linguistic magic tricks is unacceptable to anyone who understands even the beginnings of the underlying debate.
As someone who began reading the scientific literature at age 22, I accepted "non-significant" as neutral as to whether further testing for an effect would be called for. Whether it is neutral though seems case specific. In my example of a non-significant dose response effect, perhaps non-significant is too loaded against the hypothesis. The really correct way to communicate non-significant results would be through a rather long sentence referring to the power of the test, the proximity of the p-value to the criterion of significance, and to other aspects of experimental design.
Good catch!
That detail does not seem to carry any semantic weight to me? Both insignificant and non-significant are just negations of significant to me?
Im much more bothered by the repeated conflating of absence of evidence, and evidence of absence. Now in scotts defense he hardly invented this confusion and its near-universal in science journalism. But I was expecting better of him on that front, even if he is not a domain expert and he might get a lot of the details wrong.
When Alexandros publishes, I read.
I just wish he, or someone else, woild look at Ivermectin use worldwide and the scientific versus political merits. There is so much media obfuscation that prevents access to reliable data.
"Well, this took a strange turn towards the conspiratorial"
This whole pandemic has been full of scientists, public health agencies, and the media deliberately and consciously misleading the public. It's not that strange.
Thank you, Alexandros. Your analyses are brilliant, appreciated, and very important to shed light on this fraud‼️