8 Comments
Sep 23, 2022Liked by Alexandros Marinos

My first thought on hearing about the theory it was just publication bias was that when it came to Covid all rules were off. I mean many would have us ignore the huge spike in VAERS because a vaccine campaign like the Covid one was unprecedented. Likewise, what probably is a nice guide for normal times doesn't necessarily apply in the 'war time' that is the pandemic. I still would give the concern merit over in the early going but over time, once it became clear there was stigma to suggesting miracle cures unless you were Fauci talking about the game changer Remdesivir, that to tout a generic Drug would conflate you with Trump, all assumptions about publication bias should probably be tabled.

Expand full comment
author

Good point. After the FDA horse tweet, we should probably assume inverse publication bias until proven otherwise.

Expand full comment
Sep 23, 2022Liked by Alexandros Marinos

Great work. You should also be looking at dosis. My recollection is that Vallejos (which has the highest weight in the meta analysis) is badly underdosed. This is exactly the kind of thing that would create funnel plot assymetry: one underdosed large study pulling the central axis of the funnel to the right and lots of smaller correctly dosed studies whose estimated effect sizes float around the true effect size of the correct dose, which is to the left of the central axis.

Expand full comment
author

ok fine, you nerdsniped me. Here you go: https://twitter.com/alexandrosM/status/1573323696680632321?s=20

Summary, not much.

Expand full comment
Sep 24, 2022Liked by Alexandros Marinos

So apparently I was wrong about Vallejos being particularly underdosed. I haven't checked everything you did, but on a first look you seem to have made a mistake with Okumus. The dose was 200mcg/kg/day for 5 days, so 1000mcg/kg, it should be on the "high" group, not "low".

Expand full comment
author

Oh, excellent catch.

Expand full comment

By the way, Avi Bitterman made a mistake in his forest plot and funnel plot. Both plots were done using the fixed effects model, when he should have used the random effects model. I don't know how to post pictures here, so I'll send you by e-mail. It doesn't make that big of a difference.

Expand full comment
Sep 23, 2022Liked by Alexandros Marinos

Thanks; this was a topic I hoped someone would dive into a little deeper. My instinct when someone throws around 10 scattered points and starts drawing all kinds of serious conclusions has always been 'really'? But ive never taken the time for a deep dive on funnel plots, so it is insightful to see some of the thinking around such matters laid out in some detail.

Expand full comment