Demonstrating Randomization Failure in the TOGETHER Trial

I wanted to revisit the question of randomization failure in the TOGETHER trial— particularly in the ivermectin arm—because some new information has come to my attention that substantially simplifies the case for randomization failure, and also because I have never made the case in a stripped-down fashion that can be easily linked. So let’s dive right in.

1. The Timeline

The TOGETHER trial in its second iteration—starting in January 2021—initiated studies on three drugs: metformin, fluvoxamine, and ivermectin (single-dose), which was later stopped. After a few weeks, ivermectin was restarted at a higher three-dose regimen.

The three arms used a single, shared placebo arm. The timeline was as follows:

Evidence:

• (1.1) The 2021 iteration of the TOGETHER trial started recruiting patients on January 20th, 2021 (or thereabouts). More:

• (1.2) The trial initially had four arms: ivermectin (single dose), fluvoxamine, metformin, placebo. More:

• (1.3) The single dose (aka low-dose) ivermectin arm ran until some time before March 9, 2021. More:

  Together Trial @TogetherTrial

  @socialHATchat @Covid19Critical We have amended our protocol paused allocation to the ivermectin arm while the IRB reviews our amendment to include 3 doses.

• (1.4) The metformin arm ran until April 3, 2021. More:

  

• (1.5) The fluvoxamine arm ran until August 5, 2021. More:

  

• (1.6) The three-dose (aka high-dose) ivermectin arm ran from March 23, 2021 to Aug 6, 2021. More:

  

2. Patient Numbers

The ivermectin single-dose arm was terminated after recruiting 77 patients; the three-dose arm reached 679 patients. Fluvoxamine recruited 741 patients. Metformin was stopped after 215 patients were recruited. The fluvoxamine study compared the treatment against 756 placebo patients. The metformin study compared the treatment arm against 203 placebo patients:

Evidence for Patient Numbers:

• (2.1) Patients were allocated in a 1:1:1:1 ratio between the arms. More:

  

• (2.2) The single-dose ivermectin arm recruited 77 patients. The three-dose arm recruited 679 patients. More:

• (2.3) The metformin arm recruited 214 patients. Its concurrent placebo patients were 203. More:

• (2.4) The fluvoxamine arm recruited 741 patients. Its concurrent placebo was 756 patients. More:

3. Zooming In

Given that the ivermectin arm recruited 77 patients by March 9th or earlier, and the fact that patients were distributed in a 1:1:1:1 ratio to the different arms, we would expect a similar number of patients to have been recruited to each arm, with some allowance for small fluctuations due to block randomization. By August 5th, we know that number had grown to 756 patients.

Therefore, starting March 9th at the latest—and up until August 5th—approximately 679 patients were recruited into the placebo arm. That is exactly the same number of patients recruited into the three-dose ivermectin arm between March 23rd and August 6th:

So, how many patients were recruited between August 5th and August 6th?

Given that the decision to stop recruitment into the fluvoxamine and ivermectin arms was taken by the DSMC on the same day—August 5th—we should expect the database to have been locked that day, and for no more patients to be considered for either study. However, since the last day of the ivermectin trial given was as one day later, we should explore this question in more detail.

Based on the numbers from the fluvoxamine paper, the trial recruited approximately 3.8 placebo patients per day between Jan 20th and August 5, 2021 (756/198), so it would be quite remarkable if it recruited 77 patients on August 6th.

Even more remarkable is that the presentation given on the morning of August 6th— which the file metadata indicates was finalized on 5:38 AM Vancouver time (9:38 AM in Minas Gerais, Brazil)—reports 678 ivermectin placebo patients. Presumably, the data for the presentation had to be extracted, checked, processed, and imported into the slides, implying that this was data from the day before (August 5, 2021), or earlier. In short, one or two patients might have been recruited into the ivermectin placebo arm on the day of August 6th, but no more.

These dates are corroborated by the metadata of the presentation published on the NIH collaboratory website:

(source)

Also by the organization’s announcement of the event on Twitter:

(source)

Conclusion

The placebo patients recruited between the suspension of the single-dose ivermectin arm (March 9th, at the latest) and the start of the three-dose ivermectin arm on March 23rd, must have been included in the placebo group for the ivermectin study, despite the authors’ claims that the patients for that study were recruited after March 23rd.

This is a serious concern because—as detailed in correspondence published by the New England Journal of Medicine—nonconcurrent control groups are a known failure mode of platform trials—such as TOGETHER—and when they occur, they cast doubt on the randomized nature of the whole trial:

it is important to remember that the unique evidentiary status of randomized trials is based on their ability to guarantee that the trial groups have (on average) equivalent patient populations that allow between-group differences in outcomes to be attributed to the intervention. This generally does not apply to designs that compare trial groups that include patients who did not undergo randomization concurrently.

The authors also provide an illustration of how epidemiological changes in the severity of the disease can distort the results of a trial.

As the TOGETHER trial authors attested to in their metformin publication, the severity of the disease did indeed change during the course of that study, which was between Jan 15, 2021 and April 3, 2021:

The rate of our primary endpoint occurring appears to have increased importantly from the beginning of the trial to the end of the trial. This is likely explained by the emergence of the predominant Gamma (P.1) variant during the conduct of this trial that may exhibit greater transmission and worse clinical outcomes than earlier variants.

In other words, the TOGETHER authors themselves confirm what I previously conjectured: the placebo patients who were recruited earlier were less likely to trigger the primary endpoint than the ivermectin treatment patients, who were recruited at a later time.

As such, we can conclude that not only did the randomization failure occur, but by the TOGETHER team’s own account, that failure likely negatively affected the results of the three-dose ivermectin arm by comparing the treatment arm with patients that had been, in part, recruited at an earlier time—prior to the Gamma wave—when the severity of the disease was lower.

As usual, the solution here is for the TOGETHER team to release the raw data for their trial to neutral investigators—as they’ve repeatedly promised to do—so that those investigators can conclude definitively what happened, and what the impact was. If some grave misapprehension has led me astray, it should be straightforward to demonstrate by releasing the raw data.