What Went Wrong With the TOGETHER Trial?

More than 6 months later, the data from the widely covered ivermectin arm of this study has still not been shared with the scientific community.

This article is part of a series on the TOGETHER trial. More articles from this series here.

The TOGETHER trial is a trial testing multiple treatments for COVID. It's got a fascinating "adaptive platform" design that allows it to efficiently evaluate many interventions and absorb findings from global research as it goes. So far so good.

The issues I have with the trial itself relate to the ivermectin arm, not with the idea of the trial, or its design.

However, even though the slide deck with preliminary results was disseminated in early August, it's now over 6 months later, and we still don't have any more data from the trial, nor a published paper (or even a preprint), when for other arms, the respective papers were out in a few weeks. This kind of "science by slide deck" really leaves a sour taste in my mouth.

With that said, let's get into the issues:

Issue 1: No Mention of Ivermectin in the Exclusion Criteria

This one issue is probably sufficient to disqualify the study on its own, unless there has been an obvious omission in the materials shared. People taking ivermectin may have been in the control group. You'll notice SSRIs (e.g. fluvoxamine) are excluded (see criterion 7), but not ivermectin.

TOGETHER trial exclusion criteria (source)

Below is the only comment attributed to Dr. Mills, trial lead, I've found. If this is indeed his response, it's puzzling, as low community use should make it easy to add to the exclusion criteria, for avoidance of doubt.

(source)

In fact, ivermectin is easily available over the counter in Brazil, was recommended by the government, and there are reports from the time of the trial reporting 9x growth in sales around the area of the trial: (via Google Translate)

(source)

Issue 2: Gamma Variant

The ivermectin high-dose trial was executed during the time when the gamma (aka. P1) variant was dominant in Brazil. (March - June+ 2021) https://nature.com/articles/s41467-021-26459-6…

Gamma is probably the most deadly variant we’ve seen yet, so vicious that it is known to make vaccines about 25% less effective against infection and 20% less effective against severe disease. The figure below is from the Lancet publication put out by former FDA officials, highly instructive on the effect of the different variants.

(source)

This would mean that both the control group and the intervention group would be doing worse, since the ivermectin high-dose trial (the one we're discussing) was exactly on of the P.1 (Gamma) wave. (March - July 2021)

TOGETHER trial intervention timeline (source)

This issue on its own would not have been as serious of an issue if both control and treatment group were affected by the same variant, though a more aggressive variant would be expected to somewhat suppress any effect the treatment may have had. However, the next issue compounds with this one and makes the situation even more complicated.

Issue #3: Offset Control Group

Due to the ivermectin arm restarting to "high" dose, after the initial "low" dose, it seems that the control group ended up constituting of people recruited on average earlier, and in particular, highly likely included people infected with other variants than Gamma.

Why did the ivermectin arm have to restart? It seems that the original dosing had been unacceptably low, and after complaints, the trial leads agreed to restart the intervention group with higher dosing. This doesn't seem purposeful, but the virus doesn't care

(source)

Issue #4: Dosing

This issue is also compounded by the Gamma variant.

The dosing is a bit lower than what was recommended by ivermectin proponents FLCCC for the Alpha variant, compressed in fewer days, and *critically*, it wasn't recommended to be taken with a meal.

The protocol the FLCCC was offering in March 2021, (v9) was recommending 0.3mg/kg Ivermectin x 5 days with a meal.

(source)

The most important difference is recommending the treatment be taken with or after a meal. How so? Well, here's the plasma concentration differential. You'll notice it's... substantial, offering somewhere around 3 times increase in peak concentration.

(source)

Issue #5: Timing of Treatment

While we don't have definitive data (more on this later), in other arms, like HCQ and Lopinavir-ritonavir, it seems 80%+ of patients were treated more than 5 days after symptom onset. A day or two in that phase makes a big difference:

Alvaro Olavarría @AOlavarria

@boulware_dr @EdoajoEric @YangTexan @roibeard1973 Uff... you made my day! The last Together Trial had a big problem with the onset delay days. 82% > 5 days = late treatment Including a <= 3 days onset subcategory would be the best decision. jamanetwork.com/journals/jaman…

Keep in mind that in the Molnupiravir and Paxlovid trials patients were *excluded* if they were >5 days of symptom onset, meaning that the average was probably closer to 3 days from onset. A few days make a huge difference for antivirals, if we take the word of Merck and Pfizer.

Issue #6 Endpoint Not in Original Trial Design

The next issue is the odd, composite endpoint ">= 6 hr ER observation or hospitalization", which doesn't seem to have been used in other trials.

If this was the actual trial endpoint and not how they chose to report it, it would be a deviation from the pre-registration of the trial.

For more discussion on the issue with the soft endpoint, see the section “Measuring success” in this analysis of the Fluvoxamine arm of the TOGETHER trial. What is written there applies to the Ivermectin arm as well.

Issue #7: Focus On Less Severe Patients Waters Down Mortality Endpoints

There's this inclusion criterion "expected hospital stay of <=5 days". Unclear why it was needed, but it would bias to less severe cases, & combined with late treatment start, many participants would already be recovering.

For clarity, biasing towards less ill patients, or ones that are recovering anyway, would reduce the statistical significance of the result (as would the soft endpoint).

After all these issues in the trial, it's perhaps not surprising, if disappointing, that the trial was "stopped for futility".

Issue #8: Unscientific Dissemination of Results

Several of the issues so far make it so that any signal from the treatment would be lost in the noise. Not excluding patients who had received ivermectin, running the trial through the deadliest variant, using soft endpoints that include human judgement, focusing on less severe patients, and timing the treatment late, all work together to reduce the effectiveness of the treatment, or, conversely, they would indicate we need a larger study to show an effect under these conditions.

So how were the results disseminated? With the Principal Investigator describing Ivermectin as having "...no effect whatsoever...".

This is an odd way to describe effects that look like this: 9% relative risk reduction in "Extended ER observation or Hospitalization" and 18% reduction in mortality.

Now, I hear what you're saying. "It crossed the 1 line!", "It wasn't statistically significant!". I'd agree with you, if not for 2 important details. First, this is called the "bright line fallacy" and is a misunderstanding of how statistics work.

Flavio Abdenur @AbdenurFlavio

22/ This Nature paper explains among other things the basic “bright-line fallacy” of mistaking weak evidence for efficacy for (strong) evidence against efficacy:

nature.comScientists rise up against statistical significanceValentin Amrhein, Sander Greenland, Blake McShane and more than 800 signatories call for an end to hyped claims and the dismissal of possibly crucial effects.

Secondly. Look closely. Very closely. Enhance! What's that I see in the slide? Is that a... bayesian calculation claiming 76% superiority probability of ivermectin? It would really help if this finding was further explained.

In Summary

To sum up, what we know is that many things went wrong. And all of them in the direction of obscuring the effect of the treatment. Still, there was weak signal of mortality benefit. Now that we have the Paxlovid and Molnupiravir trials completed, there is no excuse to focus on trials which those antivirals would have also failed at (or at least, as per the manufacturers, the average patient in this trial would likely not have qualified for those antivirals even in principle).

Despite all this, the fact that the detailed analysis and data has not been released in over six months makes me suspect there is something inconvenient that makes the researchers hesitate. Perhaps I am wrong in this assumption, but I am yet to hear an explanation as to why the data has been delayed for this long, when the authors were quick to promote their interim results to the world with such definitive statements.

Are any of the issues I raised not as significant? Are there other issues I missed? Please let me know your thoughts on these questions or the article in general in the comments.

This article is part of a series on the TOGETHER trial. More articles from this series here.