TOGETHER Trial Variable Placebo: An Unblinding Improvisation

In clinical trials, you're not supposed to make it up as you go along.

This article is part of a series on the TOGETHER trial. More articles from this series here.

Platform trials test multiple interventions, sometimes completely different drugs, while sharing common infrastructure, such as a control group. When a control group is present, there is the challenge of how to define the placebo intervention so that it is an appropriate comparison to each of the interventions. For example, when simultaneously testing a 3-day treatment and a 10-day treatment, figuring out what the placebo should look like is a challenge. Should it consist of three days of placebo pills? Ten days? Both? Neither?

The TOGETHER trial had exactly this problem, and its approach was to subdivide the placebo group into smaller subgroups, each with a different placebo regimen, matching one of the active arms. So, when a treatment group was taking a dose every 12 hours, the corresponding placebo subgroup also took a placebo dose every 12 hours.

What Others Say About This Approach

1. Invited Reviewer

Here’s what one of the invited reviewers had to say about this feature in the Gates Open Science portal, as part of the open peer review of the TOGETHER protocol:

…the use of variable placebos in line with the agents being included is a novel and innovative approach to this issue and is to be commended.

Dr. Edward Mills responded positively, if briefly, and did not challenge the assertion that this was novel:

Thank you for the positive feedback.

2. NIH

The NIH was not as big a fan of the approach. Here’s what they had to say about this feature of the TOGETHER trial, classifying it as a “Key Limitation:”

As this was an adaptive platform trial where multiple investigational treatments or placebos were being evaluated simultaneously, not all patients in the placebo arm received a placebo that was matched to fluvoxamine by route of administration, dosing frequency, or duration of therapy.

3. TOGETHER Team

I imagined that some kind of theoretical justification may exist for this approach, perhaps something like a paper describing the potential pitfalls and what might be done to avoid them. Indeed, I found a paper by many TOGETHER-linked authors. They mention this kind of feature extremely briefly, with these words:

In placebo controlled platform trials, multiple placebos may be required in the control group.

They did, however, offer a reference. Where did the reference link? To the TOGETHER paper on Hydroxychloroquine and Lopinavir/Ritonavir, which contains no explanation or justification for using multiple placebos.

As far as we can tell, we are looking at an extremely important feature, fundamentally altering how the “C” of “RCT” is done, with no theoretical justification or explanation.

So What—Actually—is the Problem?

For simplicity, we’ll focus on the randomization process as described in the final protocol that was in force during the IVM/FLV studies, v3.0. On page 22, it gives us the parameters of block randomization used:

From this, we learn a few key details: to start, initial block size is 10. In other parts of the protocol, we can see that there are four active arms and one placebo arm. This makes sense: there are two patients per arm per block, just enough to stratify by age. Basically, something like this:

How does stratification by clinical site work though? My understanding from reading the protocol is that each clinical site makes its own blocks, therefore offering a type of stratification. Might be interesting to see the results broken down by site some day.

You might notice that there are only two placebo patients per block. According to the protocol, placebo is broken down into four subgroups, one corresponding to the regimen given to each treatment arm. Since the protocol tells us that each distinct subgroup is given a letter to distinguish them from the others, we might end up with a set of letter groups that looks something like this:

The only way I can see to fit all these groups within our block randomization scheme is to distribute patients to the placebo subgroups across blocks. So, for example, if we had four blocks that start out looking like this:

After randomization, they may end up looking like this:

At this point, the problem with the way placebo is allocated may begin to become apparent. The patient population for a clinical site that has had four blocks (or 40 patients) pass through will look something like this:

The group letter for each patient is visible on their medicine bottle and possibly in some of their documents. In other words, it’s not considered a secret. As a member of the trial staff who is nominally blinded, one only needs to see only a few patients to realize that A/B/C/D are far more plentiful than the E/F/G/H placebo patients. In addition, learning the number of days of treatment (which the patient would obviously know) the staff member can infer which intervention the patient is in. At that point, the staff member knows everything there is to know about the letter groups, and therefore about what drug or placebo each patient is taking.

In addition, since the various treatments are introduced at different points in time, the letter groups would start showing up at a distinct point in time. Trial staff can be expected to know what is happening with the trial, and therefore the appearance of a pair of letters would be pretty easy to link to the newly introduced arm and its corresponding placebo.

How Did This Feature Affect the Trial?

So far, we’ve discussed the trial design in abstract. In reality, it was even easier for trial staff to get a clear picture of which patient was assigned to which treatment or to placebo. In my prior article exposing the core randomization issues in this trial, I included this diagram:

The diagram shows allocation to each trial arm during a specific 10-week period, with one row per week. The 5th row in this diagram—the week starting March 22, 2021— had 57% of all patients randomized be assigned specifically to the ivermectin treatment arm. The placebo arm that week had only 28 of the 146 enrolled patients allocated to it, or 19.2%. Given that there were three active arms at the time, but only two different regimens (given that metformin used the same regimen as fluvoxamine), I expect that the placebo patients would have been split evenly to two sub-groups. If so, we would have 9.6% of all patients be assigned to the 3-day placebo arm, and 57% of all patients assigned to the ivermectin 3-day treatment arm. In other words, instead of the steady-state 4:1 imbalance between a treatment arm and its corresponding placebo subgroup, that particular week, the imbalance was more along the lines of 6:1. And of course, any staff that figures out the correspondence between letters and underlying treatment (or lack thereof), will have that information for the remaining duration of the corresponding arms. The eventual balancing of allocation does not unwind the information leakage.

What makes this kind of protocol feature even more surprising is that it would be fairly easy to make it harder to guess what each patient is on: using a unique code on each medicine bottle would allow the pharmacist to vend the right treatment, while at the same time allowing for unblinding if necessary (e.g. in case of the patient needing further care). The problem does not seem necessarily to be the variable placebos, but the specific implementation used.

What Now?

I really don’t know how features like this can slip through the cracks without any concern for theoretical justification. Perhaps I am missing something. If so, please let me know in the comments.

This article is part of a series on the TOGETHER trial. More articles from this series here.