The TOGETHER Files 1: The Andrew Hill connection - How the principal investigator leaked interim results to a private ivermectin research group

Leaked documents reveal the largest trial on ivermectin for COVID treatment violated its blinding protocol as the trial was ongoing.

This substack has not had an update in a while, and so the first article had to be something good. In this article you will read about the connection between the TOGETHER trial, Andrew Hill’s private ivermectin research group, and how data that has still not been released to the public was shown to a select group of ivermectin researchers while the TOGETHER trial was ongoing, in breach of data confidentiality standards.

Over the past couple of years, readers of this Substack have followed a deep dive into TOGETHER, the largest randomized controlled trial (RCT) set up to investigate the potential of repurposed generic drugs including ivermectin as an early treatment for COVID. Over the course of this investigation, we have been shown many documents that were leaked, confidential, or otherwise private to the trial. We have kept them secret until now to give sources time to apply pressure to the investigators of the trial release their data. Since that clearly is not happening, the time has come to release everything we have gathered so far to the public, true to the spirit of this Substack.

For our first article in the series, we will expose the connection between the TOGETHER trial, and Andrew Hill, the infamous meta-analysis author who was caught on video confessing that he modified the conclusion of his meta-analysis on ivermectin at the behest of his funders, UNITAID. The change explicitly discouraged regulatory authorities from recommending ivermectin for COVID, despite the initial meta-analysis finding a 75% reduction in mortality. Needless to say, allowing such influence without acknowledgement constitutes textbook academic misconduct at the very least.

For those new to this Substack, the TOGETHER trial for ivermectin, which ran from January to early August 2021, served as one the last nails in the coffin of ivermectin for COVID treatment. Six months before TOGETHER even published its formal journal paper, the BBC cited it when it wrote that no evidence had shown ivermectin to be efficient against COVID. The study received accolades, and that same BBC article won a prize for exposing false information.

But how reliable are TOGETHER’s findings for the ivermectin study? Two years after their initial promise, the authors still haven’t released the raw data — even to their own funders, close collaborators or meta-analysis authors, even though the authors have made grand statements about data sharing in the pandemic emergency. 

More recently, leaked emails and documents show that TOGETHER’s principal investigator Edward Mills breached key rules on data blinding and confidentiality. Importantly, there are serious indications that this improper access could have enabled the investigators to make decisions that would affect the results of the trial.

Before going any further, we should say our concern here is not simply whether ivermectin helps in treating COVID, or not. That is an important question, but one which pales in comparison to the unaccountable corruption of clinical trials writ large. Our focus is on how a very influential RCT on early treatment for COVID appears riddled with ethical and scientific issues, with but few calling it to task. In publishing our latest findings here, we hope to press the authors to provide answers, and encourage anyone with further information to reach out, either by DM on X, or here in the comments below.

TOGETHER investigator accessed, and shared, unblinded data

We started digging into TOGETHER in the spring of 2022. In the process, we’ve uncovered numerical aberrations and ethically dubious conduct. We’ve also uncovered funding from the Bill & Melinda Gates Foundation, despite it not being listed among the study’s funders or conflicts of interest.

Perhaps the most troubling finding is the one we’ll detail here. It stems from leaked documents showing the TOGETHER trial presented detailed interim results even as the study was ongoing, against their own protocol for data blinding.

This is made clear from email invitations and Powerpoint presentations we’ve secured, from a series of meetings organized by Dr. Andrew Hill during 2021 under the title “International Ivermectin Project Team”.

The project invited worldwide investigators of ivermectin for COVID treatment to regular meetings, where they could share information and interim results from their trials. These meetings happened every few weeks, from early 2021 until early 2022. Information about these meetings has not reached the public before to our knowledge, and how Hill used them to steer the narrative on ivermectin is a chapter that still needs to be told.

On May 25, 2021, the project’s then 91 members received the following agenda for the upcoming meeting:

Notice this item:

Interim analysis of the TOGETHER trial in Brazil – Dr. Edward Mills, Senior Principal Scientist, Cytel

Note that in May, the TOGETHER ivermectin study was still ongoing — recruitment didn’t officially end till August 6. Sources present at the meeting have shared with us the slides that Mills presented that day. They show results of an interim analysis, as well as day-by-day enrollment numbers by treatment arm. These were not supposed to be seen by any blinded staff of the trial, yet not only were they accessed, they were shared externally.

Some of those results — such as the ones in the low-dose ivermectin arm — are yet to be released publicly even today. We will publish the slides and those results in this series as soon as we are able to. We have also seen the next meeting’s agenda, indicating that a similar presentation took place on July 2, 2021:

The title of this talk, “Randomized trial analysis of Ivermectin in 2400 patients in Brazil”, makes it less clear whether actual results were shared, and we have not gotten materials or testimony about the contents of this presentation. It is worth noting, however, that like the previous meeting, Andrew Hill presented updated results of his running meta-analysis on the drug. It is very concerning to know that while major trials were running, many of the principal investigators were getting privileged information as to the state of the aggregate data from the other trials. We can’t think of many ways in which this could help the truthfulness of their conclusions, but it’s easy to think of ways in which such a process can inject bias. The very idea of a meta-analysis relies on the fact that the trials are independent, and these meetings seem to have acted as a way to cultivate groupthink.

The very presentation of that data shows that several internal firewalls in the TOGETHER trial, as defined by their own protocol, either did not exist or had failed. (We’ve also seen emails that indicate these results were shared at the time with higher-ups at NIH. We are not at liberty to share those emails, at least for now, but hope to explore this in the future as part of the TOGETHER files.)

For context, large-scale double blind randomized trials are expected to have several steps in place to safeguard data confidentiality and blinding. Blinding study researchers is done to minimize bias in the study, and in turn minimize the ability to tamper with results. Typically, only the trial statistician can access unblinded results, which they use to prepare blinded reports for the Data and Safety Monitoring Committee (DSMC). The committee exists specifically to prevent the principal investigators from accessing interim trial results, even in blinded summary form. And, of course, in no case should interim results be shared outside the trial.

The paper “Maintaining confidentiality of interim data to enhance trial integrity and credibility“ summarizes the consensus positions that confidentiality must be taken extremely seriously:

‘There is near unanimity that the interim data and the deliberations of the DMC should be absolutely confidential … At the end of the meeting, the DMC will make a recommendation to the steering committee, but the DMC will not discuss the actual data with the steering committee or anyone else. Breaches of confidence are to be treated extremely seriously.’

In this case it’s not just that the results were shared outside the DSMC, but they were disseminated outside the trial itself.

TOGETHER made major changes part way through the trial

So we know that the data confidentiality that is expected of professional clinical trials was breached. That still leaves the question of whether this breach had any effect on the results of the trial.

One way to find out is to see if any changes were made to the protocol that have been informed by interim results, and would have affected the study outcome.

Examining the various versions of the trial protocol, the answer to both these questions is yes. In particular, TOGETHER made fundamental changes to its inclusion criteria part-way through the trial, as the ivermectin and fluvoxamine arms were ongoing.

On May 21, 2021, the authors filed an updated protocol for recruiting patients. The change, approved by Brazilian authorities four days later, added the following “enhancement factors” to the trial:

• Patient with fever thermometry at screening > 38º C.

• Patients with at least one of the following symptoms: Cough, Dyspnea, ventilator-dependent chest pain or myalgias with limitation of daily activities (Criterion limited to 25% of randomizations).

If we look for the justification—in the version history of the master protocol—they write that…

If you have a cough, or a fever over 38C (100.4 F) you are at high risk for complications due to COVID-19? This aberration contradicted TOGETHER’s very own protocol, which listed dry cough, fever and dyspnea as the most common manifestations of symptomatic COVID.

The change is even more jarring considering TOGETHER was created to investigate treatments for high-risk patients. Per its own protocol, TOGETHER was set up to evaluate the efficacy of drugs “… on reducing hospitalization in patients with mild COVID19 and high risk for complications”. With this unjustified midstream change, the entire protocol is upended, as the vast majority of all patients presenting with symptomatic COVID-19 would qualify for inclusion.

The move also contravened FDA guidance, which states inclusion criteria must remain constant throughout recruitment, with a few exceptions, none of which the TOGETHER trial met. Should a criterion be changed, the guidance says the study protocol shall be updated to cover any resulting statistical changes. There is no indication the statistical analysis plan was updated with any adjustments to address the changes in inclusion criteria, exclusion criteria, randomization algorithms, etc. Not only wasn’t it done, the final paper presents only one version of the inclusion criteria, giving the reader the impression there were no material changes throughout. While some versions of the protocol are attached in the appendix which contradict what is written in the paper, those will not be read by the vast majority of the audience.

Interestingly, when the authors declared the protocol change to clinicaltrials.gov — the non-editable government database — they left out “cough” as a symptom from the new enhancement criterion. Perhaps it was an honest mistake. Or perhaps they didn’t want to attract attention to how trivial it was to now qualify for the trial.

Those with a keen eye would also have picked up another curious detail in the updated protocol: Criterion limited to 25% of randomizations. We have not seen any similar limits in other trials, but if someone wanted to water down the statistical significance of a patient pool, inducting a certain fine-tuned amount of patients who have no significant risk factors and are almost certainly not going to be hospitalized or visit an ER is the sort of thing that they would do. And unfortunately the authors don’t offer us any justification whatsoever for such a drastic yet odd change in the middle of the trial. Also of note, the FDA has raised concerns about the “conduct” of the TOGETHER trial, though the content of these concerns are not known so may or may not be relevant here.

Another explanation is that since the trial was based on the assumption that about 15% of patients will have an “event” (hospitalization or observation at an ER for over six hours), you might not want it to exceed that amount. Of course, trialists are not supposed to continuously fine-tune trials so that they prove themselves to have been correct, retrospectively. And none of these are good reasons to modify inclusion criteria en route: they are fingerprints of statistical manipulation.

In fact, given that Cytel—the company that “designed and led the TOGETHER trial”—bragged about its Clinical Trial optimization simulation software, it would be good to feel like someone wasn’t running interim data through simulations to figure out which criteria would give them the desired results.

What about the DSMC, you might ask, shouldn’t they have pushed back on such blatant violations of the trial protocol? You would expect so, except that two of its members including the chair work for Cytel, and most members share academic history with Mills, making the committee’s independence questionable at best.

New protocol likely skewed results against ivermectin

Now, to the question of impact.

By including patients with milder symptoms in their inclusion criteria, TOGETHER’s new protocol increased the likelihood of younger people being included in the trial. Looking at the data available, this seems to have happened. To understand how we can know this, a word about how TOGETHER works. TOGETHER is an adaptative trial platform. This allowed it to evaluate fluvoxamine, metformin and ivermectin concurrently between January and August 2021, using — and this is the key part — a *shared* placebo group.

Helpfully, in the journal publications, both the metformin and the fluvoxamine studies report the make-up of the placebo groups by age, and all three arms started at roughly the same time. Because the metformin study was stopped early (on April 3rd) for futility, we can compare how these numbers changed after April.

The metformin study reported 203 placebo patients enrolled by April 3, of which 120 (59%) were over 50. By August 5, the numbers were 328 out of 756 according to the fluvoxamine paper. Using subtraction, one can deduce that only 37.6% of placebo patients enrolled between April 3 and August 5 were over 50.

In other words, more than half (59%) of people enrolled in the trials in the first 3 months were over 50. But between April and August, nearly two-thirds of new recruits were under 50. That is a quite a drop.

Now if we look at the published papers, we see something very interesting. The TOGETHER paper on ivermectin indicates that any benefit of the drug was exclusively in patients over 50.

In contrast, the fluvoxamine study indicates the anti-depressant did best in those under 50.

In other words, the change in age ratio would have watered down the effect of ivermectin, and boosted that of fluvoxamine. As a side note, subgroup analyses should typically only be used for hypothesis generation, and not as standalone results. However, the patients in TOGETHER were stratified by age (over/under 50), meaning the subgroups were randomized separately. As such, the results can be interpreted as valid standalone results. That said, due to the many versions of the protocol and statistical analysis plan that were published during the course of the ivermectin arm, it is unclear whether the stratification applied throughout protocol versions, another area of concern.

Wrapping it up

So, what can we take away from these findings?

We know that interim results were shared outside the TOGETHER trial, while the trial was ongoing. This is in violation of the protocol, as well as established norms around confidentiality and blinding in clinical trials. Once these results leave the walls of the trial, outside influence and interference with the trial cannot be ruled out.

It’s hard to make precise conclusions about how this release affected the results without direct access to the raw data of the trial, but what we do know is:

• Interim unblinded results from the TOGETHER trial were shared with a group of dozens of researchers of other ivermectin trials while the trial was ongoing.

• There are indications that these results were shown to higher-ups at NIH.

• Shortly afterwards, the authors made a significant change to the protocol for enrolment. The change — which was made in contradiction of FDA guidelines — made it more likely that younger and less ill patients would be recruited. Indeed, the trial demographics reveal a strong shift towards more placebo patients under 50.

• Had this change not taken place, we should expect that the final results would have been more favorable towards ivermectin, and less towards fluvoxamine.

• While it’s hard to prove intent, the bias introduced here is the same bias as seen in a number of other areas of the trial. The randomization failure in the earlier stages of the trial was favorable to fluvoxamine, at the expense of ivermectin. Similarly, the exclusion criteria throughout the trial excluded patients taking SSRIs (similar to fluvoxamine) but not patients taking ivermectin. Finally, while ivermectin patients were underdosed, in conflict with the trial’s own rationale, fluvoxamine was administered at a relatively high dose, twice a day, for 10 days.

This fact pattern is entirely consistent with an intentional change of the trial’s inclusion criteria, informed by access to interim analyses, with the expectation that it would lead to better results for fluvoxamine, and worse results for ivermectin.

If the authors have a better explanation, they can certainly tell the world. Even better, access to raw data from the trial should demonstrate conclusively if the change in criteria positively affected the results of fluvoxamine, at the expense of ivermectin.

If you have more information about the conduct of the TOGETHER trial or Andrew Hill’s “International Ivermectin Project Team” please leave a comment below or contact @alexandrosm over direct message on X (formerly Twitter).