Fatal or Harmless? It Depends

If you want to understand how dangerous open science can be, look no further than preprints, which routinely show why scientists shouldn’t be allowed to post information before it’s ready for public consumption.

This occurred during the early days of the Covid-19 vaccines when physicians from the Ottawa Heart Institute posted a preprint on medRxiv in which they estimated the incidence of myocarditis after mRNA vaccination for Covid-19 occurred at a rate of approximately 1 in 1,000. This lit a firestorm of alarm across the anti-vax world, one which burns to this day.

Sadly, despite the data being publicly available and the math simple, these “smart” people made a dumb calculation mistake — there were ~25x more vaccines administered than they accounted for, making the risk 1 in 25,000, or lower than the risk of myocarditis from Covid-19 itself.

Since then, other, better studies have shown that people receiving Covid-19 vaccines have a lower risk of cardiac death overall.

While the preprint was “withdrawn” relatively quickly, the shockwaves from its detonation were only beginning, just as those with the “withdrawn” ivermectin preprint from early that same summer took a long time to subside — after an unknown amount of sickness and death.

This was a version of open science — publishing findings as they accumulate, letting people see “the sausage being made.”

Yet, what that can do if the scientists are in a hurry or excitable is fuel conspiracy theories and spread misinformation. There are always bad actors around, sniffing for something provocative. Open science advocates are too naïve to know this.

Now, on top of preprints having become a form of Green OA and a fertile ground for corporate PR disguised as science, we have another example of why open science via preprints is a bad idea.

Saturday, KevinMD (“Social media’s leading physician voice”) posted about a preprint on bioRxiv claiming to have identified a variant of the SARS-CoV-2 that “can cause 100% mortality in human ACE2-transgenic mice,” with “a spillover risk . . . into humans.”

Sounds scary? Another SARS-CoV-2 virus that can kill every mouse it touches and could infect humans with the same devastating effects?

Well, that version of this very short and superficial (described as a “letter to the editor”) preprint went up on January 4, 2024. It is still available, and indexed for search like anything else. But two weeks later, another version was posted with some interesting further findings (KevinMD had access to both versions, but of course chose the most alarmist to highlight):

• The headline changed
  • First version: “Lethal Infection of Human ACE2-Transgenic Mice Caused by SARS-CoV-2-related Pangolin Coronavirus GX_P2V(short_3UTR)”
  • Second version: “An infection and pathogenesis mouse model of SARS-CoV-2-related pangolin coronavirus GX_P2V(short_3UTR)”
• The researchers no longer claim their findings are generalizable because the mice they were studying are atypical in bizarre and important ways, and there isn’t any risk of human spillover infection
  • “. . . we are notified that these hACE2 mice have abnormal physiology, as indicated by relatively reduced serum triglyceride, cholesterol, and lipase levels, compared to those of wild-type C57BL/6J mice. Thus, the outcomes from the mouse infections in this study have no correlation with human infections”

So, there you have your open science for the day — a preprint posted with little data and dangerous, misleading claims; amplification through social media and misinformation vendors; and, uptake in the popular press.

This isn’t the first time a preprint using mouse models has caused trouble. In 2022, Boston University researchers released a preprint (which they later revised heavily) claiming 80% mortality in certain mice for a modified SARS-CoV-2 virus they were playing with. Their results have never been published in a peer-reviewed journal, but debunking continues to occur, as with this latest example.

Why we allow this kind of incomplete and unreliable science to go to scale via preprints is beyond me. Biomedicine was always a third-rail for preprints, and we’re seeing again and again why many of us thought it was a no-go zone for unreviewed claims and preliminary findings.

Open science in the health zone is dangerous because, frankly, too many scientists rush their experiments, take shortcuts, are excitable, are bad at math, and/or are eager for the spotlight.

We’re here to slow them down, help them put information into the public sphere the public can trust, and ensure the scientific claims they are making as robust and reliable as possible, given who scientists are (fallible humans who often don’t have a lot of common sense) and how scientists work.

Preprints in biomedicine continue to be a bad idea. Nothing is going to change that.