10 Questions for the TOGETHER Trial Investigators
In which I attempt to boil things down to their core essence.
This article is part of a series on the TOGETHER trial. More articles from this series here.
I’ve written quite a bit of material on the TOGETHER trial, and a lot of it is hard to parse for someone who has not been following along. This article is a distillation of questions that are both fairly easy to articulate, and should be fairly easy for the investigators to answer, if a straightforward answer exists.
Question #1 - Randomization Anomalies
In the slides released by the investigators on August 6, 2021, the following enrollment chart was shown:
It can easily be seen that the last two weeks of March 2021 contain an outsized allocation of patients to the high-dose ivermectin treatment arm, while all other arms are underallocated. In fact, by superimposing a grid, it can be seen that in the week of March 22nd—the first week of high-dose ivermectin recruitment—57% of all patients were randomized to the high-dose ivermectin treatment arm. The next week, 41% of all patients were randomized to the high-dose ivermectin treatment arm, as well as 45% of all patients the week after. This kind of allocation asymmetry can lead to many issues in platform trials, as described by a recent article in the New England Journal of Medicine.
Given that block randomization doesn’t normally allow for such an imbalance to arise, how did this allocation anomaly occur, and what was done to adjust for its effects?
Read more about this issue here.
Question #2 - Dosing
The dose delivered did not scale beyond 90kg, as can be seen in the protocol, a fact not mentioned in the paper. Given that almost half the patients were at 30 BMI or higher, it is quite obvious that about half the male patients, and a substantial amount of female patients, were treated below the claimed dose. Since BMI is a significant risk factor for COVID-19, the dosing gap was more extreme as patient BMI—and therefore risk—increased. The justification for dosing written in the protocol not only does not mention any weight limit to dose scaling with weight, but in fact recommends dosing at 400mcg/kg with no caveats whatsoever.
Why was a dosing limitation at 90kg implemented, and why wasn’t it mentioned in the published paper?
Read more about this issue here.
Question #3 - Exclusion for Prior Use of Ivermectin
The new "limitations" section in the research summary—updated as of May 5, 2022—of the published TOGETHER ivermectin study states that, "the authors attempted to screen potential participants for previous ivermectin use.” This contradicts the discussion section of the same paper that states, “We ensured that trial participants did not have a history of ivermectin use for the treatment of Covid-19 by means of extensive screening of potential participants about this issue.” This contradiction should have been resolved clearly by the exclusion criteria of the study, however there is no indication that prior use of ivermectin would exclude a patient from the study, nor does the screening questionnaire include such a question. Reports from the area indicate 9x increase in ivermectin sales in the region at the time of the trial, so there is no question the drug—which is available over the counter there—was in common use, something the paper seems to agree with. The difficulty caused for clinical trials in South America by the drug’s popularity has been reported on by high-profile publications, yet the paper seems to not adjust or account for this.
Why was there no clearly stated exclusion criterion for ivermectin use? How was the exclusion for prior ivermectin use conducted? Can the authors quantify the potential impact background use in the population had on the final results?
Question #4 - Variable Placebo
The subdivision of the placebo group into smaller subgroups corresponding to each treatment arm (1-day, 3-day, 10-day, 14-day,...) was praised during the open peer review of the protocol:
…the use of variable placebos in line with the agents being included is a novel and innovative approach to this issue and is to be commended…
However, the authors cite no theoretical justification for using this scheme, especially given the fact that the resulting uneven allocation of patients to different letter groups allows the trial staff to infer which patient is on which arm—or which specific placebo subgroup—with a small number of data points, thus unblinding them. This is especially concerning, since the ER observation component of the primary endpoint involves subjective judgment of potentially unblinded medical personnel. In addition, it is unclear why the alternative of a unique code on each medicine bottle was not used, which can serve the same purpose without the downside of unblinding.
Why was the variable placebo approach introduced? What was done to correct the potential unblinding of the investigators and other clinical staff?
Read more about this issue here
Question #5 - Novel Endpoint
The trial used a novel endpoint of “ER observation > 6 hours, or hospitalization.” Not only is this composite endpoint an innovation of the trial designers, it is an endpoint that was changed about two months after the trial was started. Later, the results were presented as a singular endpoint, even though the straightforward reading of the protocol would necessitate separating out the two sets of numbers.
Why did the authors adopt this novel endpoint? Why did they combine the results of the two separate endpoints into one for all the reporting they did after the trial ended?
Read more about this issue here.
Question #6 - Missing Data
Figure 2 in the ivermectin paper reveals that there is a significant amount of missing data. In particular, it seems that “time since onset of symptoms,” is missing data for 317 patients, which is 23.3% of patients. Similarly, it appears that age information is missing for 98 patients, or 7.2% of the whole. Both of these are essential inclusion criteria.
How did the authors ensure the patients were qualified to be included in the trial if that data was not collected? Why was this data imputed in Table 1 since, especially for time-since-symptom onset, it exceeds the 20% threshold defined in the Statistical Analysis Plan?
Question #7 - Dose Change and Enrollment
It is unclear what series of steps were followed to determine the change in dosing for ivermectin. In particular, the revised protocol is dated February 15, 2021, and it was submitted to the Brazilian authorities on February 19, 2021. Yet, recruitment into the low-dose arm continued for weeks after that point.
What was the process followed to decide the change in ivermectin dosing? Why didn’t the low-dose ivermectin arm stop recruitment as soon as the decision was made to move to a higher dose?
Question #8 - 3-Day Placebo Adherence
In the TOGETHER ivermectin paper, the 3-day placebo subgroup is described as containing 288 patients. Unlike the fluvoxamine paper, we are not told how many of these patients did not fully adhere to the protocol, but it seems that all of them are included in the per-protocol analysis, even though the treatment group is reported to have had 8.1% dropoff. The only other reasonable option is for the 288 number to represent patients after the dropoff had taken place. However, that would require the initial number to be more than 300 patients. Any number meaningfully greater than 288 constrains the population of the other placebo subgroups impossibly.
How many patients were assigned to the 3-day placebo group, regardless of adherence? What method was used to allocate patients to the different placebo subgroups?
Read more about this issue here.
Question #9 - Independent Monitoring Board Not Independent
The Data and Safety Monitoring Committee (DSMC) is a key decision making body in this trial. The various publications clearly state that it is independent of the investigators. Its independence is paramount to the trustworthiness of the trial, given that it is unblinded and makes decisions about when arms should be started or stopped. However, the chairman of the DSMC, Dr. Kristian Thorlund, has over 100 shared publications with one of the principal investigators—Dr. Edward Mills—works at the same company as Mills—Cytel—and started a prior company together—MTEK Sciences. Also, he teaches at the same university—McMaster. Dr. Thorlund can clearly be seen described as a co-lead of the trial with Dr. Mills in the earliest versions of the TOGETHER website, and is a co-author on the study protocol.
Further, Dr. Jonas Haggstrom, also a member of the DSMC, worked with Mills and Thorlund at MTEK and then followed them to Cytel after MTEK was acquired. Dr. Sonal Singh, another member of the DSMC has more than 25 shared publications with Dr. Mills.
Given the number and strength of connections between Dr. Mills and several members of the DSMC, how can it be considered independent? Why was Kristian Thorlund not removed from the committee when this was noted as an issue by an invited reviewer during the open peer review process in August 2021?
Read more about this issue here.
Question #10 - Failure to Share Data
The entry of the TOGETHER trial in clinicaltrials.gov proudly proclaims that individual participant data (including patient tables and main data) will be shared as of protocol termination, upon request.
In August, with the publication of the study protocol, the following paragraph was included:
With the publication of the TOGETHER study on fluvoxamine on October 2021, in The Lancet, the following data sharing statement was attached:
As of December 2021, with the publication of the TOGETHER study on metformin in The Lancet, the following data sharing statement was attached:
As of March 2022, with the publication of the TOGETHER study on ivermectin in the New England Journal of Medicine, the following data sharing statement was attached:
Even though there are five distinct commitments to release the study’s data—some with hard deadlines attached—as of May 2022, there has been no external access to the dataset to my knowledge. Many requests for access to the data have gone without answer.
Do the TOGETHER investigators intend to keep their original promise, still present on clinicaltrials.gov, to release individual participant data upon request, with no further strings attached? Why have prior commitments to provide access been broken? Given the significant concerns about the results of the study, will the authors agree to an independent audit of the study’s data?
The End
I hope this article has served its purpose as a repository for straightforward questions the investigators could choose to answer, if they so desire. Their continued silence is adding to the already heightened tension around these issues. By providing answers and data, the authors can help clear up any mistaken impressions.
This article is part of a series on the TOGETHER trial. More articles from this series here.
Did this abomination really just win an annual best study award?
Thank you! very helpful and very shareable with others!