TOGETHER Trial: Impossible Numbers and How "Independent" Monitoring Failed Us

This article is part of a series on the TOGETHER trial. More articles from this series here.

Summary:

1. The recently published ivermectin trial from the TOGETHER trial authors claims that placebo and treatment patients had been randomized from March 23rd onwards. This article demonstrates that this could not be true, or at least contradicts other published statements by the same authors. The most likely explanation is that 75 patients had been included in the trial earlier than March 23rd, and most of them earlier than the official approval of the trial by the Brazilian ethics board. Besides the ethical issues, this creates issues with the randomization and blinding claims of the study.

2. The application for ethics approval reveals that the authors had decided on the termination and replacement of the earlier low-dose ivermectin trial by February 15th at the latest. However they kept recruiting patients, randomizing 59 cases to the low-dose trial after they had determined to halt it.

3. These issues should have been caught by a Data and Safety Monitoring Committee (DSMC). However, this committee is not independent in the case of TOGETHER, staffed with people deeply involved in the design and with financial ties to the company behind the trial.

The above claims will be illustrated in the following three parts:

Part 1 - Impossible Numbers

The TOGETHER trial is an Adaptive Platform Trial representing an elegant concept, rooted in bayesian statistics, which aims to make maximal use of information while minimizing the number of patients that have to be at risk to reach conclusions.

The trial began with testing hydroxychloroquine as well as Lopinavir/Ritonavir in the latter half of 2020. Recruitment for this first epoch had wound down by September 30th.

(source)

Enrollment restarted in January 15, 2021, for three more medications: ivermectin, Metformin, Fluvoxamine. This is where things get odd.

According to the paper, eventually published for the Fluvoxamine arm, there were 756 patients allocated to placebo between Jan 20th and August 5th. The only other placebo patients mentioned are the “227 previous placebo” patients, who correspond to the 2020 hydroxychloroquine epoch. All other patients described were assigned to some treatment arm.

(source)

According to the paper published for the concurrent high-dose ivermectin arm, there were 679 placebo patients, and the high-dose ivermectin trial was terminated by a decision of the TOGETHER Data Safety and Management Committee (DSMC), on August 5th. It appears that there were patients randomized on August 6th, but due to the decision to terminate on August 5th, I assume any additional patients were minimal as a result of slow communication to the various sites.

(source)

The ivermectin paper clearly states:

The evaluation that is reported here involved patients who had been randomly assigned to receive either ivermectin or placebo between March 23, 2021, and August 6, 2021.

This, however, presents a conundrum. For everything we know to add up, only 77 placebo patients (756-679) must have been randomized between January 20th and March 23rd.

The confusion intensifies when we take into account the paper on the third drug tested in parallel: Metformin. The paper says that patients started being randomized on January 15, 2021, and patients stopped being included on April 3rd. Given that the the local ethics board in Brazil would not grant approval until January 18th, I assume that few if any patients were recruited before then.

(source)

If 203 placebo patients had been randomized until April 3rd, as the Metformin paper indicates, but only 77 patients had been randomized to placebo before March 23rd, as indicated above, that would mean that 126 patients were randomized to placebo within an incredibly short window: March 23rd to April 3rd, or in other words nine working days, within which 62% of the placebo patients of Metformin would have to have been randomized.

This becomes even more concerning when we see what was happening exactly on those 12 days: the absolute peak of the Gamma variant wave. The Gamma variant is considered 1.5 times more likely to result in death than previous variant.

(source)

The peak of deaths occurs on April 8th, with 492 deaths reported in a single day, the worst of the pandemic. Naturally, the people who died on April 8th would have been having symptoms two weeks earlier, right around March 25th.

We either have to believe that something incredibly cynical happened, with placebo patients for all three trials being recruited during the most chaotic and lethal time of the pandemic in Brazil, or we’re missing something.

Let’s Dig Deeper

The researchers made an incredibly helpful graph available to us on August 11th, as part of their presentation of interim results:

(source)

This starts to give us some hints, but ideally we’d need some numbers. Thankfully, we can overlay a grid and extract some actual digits.

If we count, we end up with the following numbers:

Many thanks to @Volishun for the hard work to put this dataset together.

The first thing that becomes apparent, is that the chart is not complete. The last week is the one beginning on July 26, 2021, and it very well may be that the last few weeks have incomplete data. One more thing that should increase our confidence in this chart and our method, is that the numbers for hydroxychloroquine and Lopinavir/Ritonavir, as well as the 2020 placebo patients, match exactly.

Given that the presentation took place on August 11th, it’s quite likely that not all the most recent data had been added.

From the Fluvoxamine paper, however, we can quantify what data is missing.

First, since the graph shows us 713 Fluvoxamine patients, there are an additional 30 patients randomized to the same arm some time in the last few weeks.

Second, since the graph shows us 711 placebo patients in 2021, there must have been an additional 45 placebo patients until August 5th to reach the 756 patients described in the paper.

Here’s where things become tricky though: if these numbers we have are accurate, then we only see 559 patients randomized to placebo after March 22nd, the start of the high-dose ivermectin arm. If we add the extra 45 placebo patients that we inferred from the Fluvoxamine totals, we still only have 604 placebo patients recruited after the high-dose ivermectin arm started.

And yet, in the ivermectin paper we see clearly stated:

(source)

Not all these facts that have come from the TOGETHER team can be true, and the simplest explanation is that the ivermectin placebo arm includes 75 patients randomized before March 23rd. This kind of offsetting compromises the results of the analysis greatly, especially when we take into account that the Gamma wave was building momentum, and hit its peak just as allocation to the treatment arm began.

Part 2 - Ethics Approval Applications Tell a Story

We already saw how the trial started before the local ethics board in Brazil gave approval for the 2021 studies. However, the approval for the updated protocol gives us a hint as to why the authors would insist all ivermectin patients were recruited and randomized after March 2023. The approval was granted on March 15th. So any patients recruited earlier would be, again, in violation.

(source)

That document tells another story too though, one that I find far more objectionable. It turns out that the update to the protocol had been requested since the 19th of February, and the new protocol was dated February 15th. Presumably preparations to stop the low-dose ivermectin arm and restart it at a high dose were ongoing even earlier. And yet the DSMC did not pause the low-dose arm, even though the investigators were convinced the dose was too low to have an effect, and the data from the trial was not going to be used anyway.

(source)

But this isn’t what happened. Instead, they kept recruiting patients to a trial they knew was going to be halted, and a submission to the local ethics board was pending to start over. If we take the 15th of February as the starting point, we see that only 19 patients had been randomized to low-dose ivermectin at that time. The other 59 patients were included AFTER the protocol had been revised.

Why did the investigators recruit patients for a trial they had concluded needed to be stopped?

Part 3 - Where Was the Data and Safety Management Committee?

Every version of the TOGETHER trial protocol available contains this paragraph:

So who is on the DSMC for the TOGETHER trial? The ivermectin paper’s supplemental appendix gives us some names:

Doing a basic check, James Orbinski seems to have co-written nine publications with Edward Mills, the mastermind behind the TOGETHER trial. That sounds significant, but perhaps it's OK.

Sonal Singh has a more substantial co-authorship connection with Mills, having written 29 papers together.

What about Kristian Thorlund?

They've literally written over 100 papers together. They really seem close.

The Wayback Machine gives us the clue we need. When the trial started, in the first version of the website, this FAQ refers to Mills and Thorlund as joint leads of the project.

I’m not sure how Thorlund can be claimed to be independent at all.

There's something else that's strange about the first version of that page, though. It points all emails to a company I haven't heard before, called MTEK Sciences.

Searching for that name shows a lot of very interesting material. This page here has quite the trove, including two grants from the Bill & Melinda Gates Foundation.

However it is this paragraph that is most familiar:

Looking into the concept of the Highly Efficient Clinical Trial, this paper shows up:

A few things of interest: MTEK employed both Thorlund and Mills. One more relevant name: Jonas Haggstrom. He is also part of the DSMC of TOGETHER. Finally, both Mills and Haggstrom were simultaneously affiliated with the Bill & Melinda Gates Foundation, who funded the TOGETHER trial.

Trying to understand what happened to MTEK, I hit the jackpot: Acquired by CYTEL in 2019, Mills as well as Thorlund are referred to as "Founding Partners and Directors.” It's basically their startup. I can't help but wonder if MTEK stands for "Mills Thorlund Edward Kristian.”

Given all that, I'm not sure how this DSMC can be considered independent. Thorlund seems to be just as invested in the success of the research protocol as Mills. For all we know, they both could have contractual terms in their acquisition agreement that relate to the performance of MTEK products, such as the technology behind the TOGETHER trial. These are quite common. Haggstrom also seems to be very tightly linked (and still employed by CYTEL).

Surely someone noticed? Actually, someone did. In the open peer review of the protocol, this very issue was raised by a pair of Danish reviewers:

Mills responded to their concerns thusly:

The reviewers did not budge and did not give the protocol their full approval, since Mills refused to remove Thorlund as chair of the DSMC and only took his vote away. Why would they insist on having Thorlund on an "independent" Data and Safety Monitoring Committee?

The authors give their final response with two references: One is the trial design paper with Mills and Thorlund as co-authors, and the other is literally a book on Data Monitoring Committees in Clinical Trials.

It is stunning that the authors not only appointed a clearly non-independent DSMC for their trial, they actually more or less ignored direct reviewer advice pointing this out. And yet they insist on repeating this paragraph on every version of the protocol that the DSMC will have no involvement.

Data Must Be Released Immediately

Given the incredibly troubling numbers emerging from the trial, we would really need to see more data to understand what is going on.

The authors had committed to release data to interested researchers upon request as soon as the protocol terminates. Every version of their registration on clinicaltrials.gov includes the following:

Sadly, now, more than six months after the completion of the trial, and many months after the publication of prior results, they are suggesting they will eventually give the data to a connected third party who will adjudicate access based on proposals submitted by interested researchers. And while their data sharing statement to NEJM states that this process can at will begin immediately, the principal investigator now suggests the relevant people are busy and this release will happen some time in the future:

There is a very long list of other irregularities and protocol violations that I have not included in this article, but will discuss in future installments if need be. The best compilation of issues is here, though it includes a few items that I think have good explanations and aren’t actual concerns. This article is focused on the impossible numbers emerging around the placebo group, unethical practices, and the lack of independence of the DSMC, whose job it is to ensure such irregularities do not occur.

Conclusion

The refusal to remove Thorlund as chairman of the Together DSMC, as well as the evasiveness with regard to access to data as promised in the original registration of the trial, especially in light of the irregularities emerging from the analysis of what data has been released leaves me extremely unsettled. Could the largest trial of repurposed early treatment drugs, leading to 4 publications in major medical journals, have been done this badly?

If there are any errors in my analysis you would like to note, or other comments relating to this article, please leave them below. My primary concern is to be precise in the statements included here so any corrections are greatly appreciated.

Before we wrap up, I want to acknowledge @MaryKRe and @mmpaquette for input and insights relating to this post, as well as @mightycanoe with editorial support.

This article is part of a series on the TOGETHER trial. More articles from this series here.