Preprints and Journalists Take Over

A recent editorial in the Wall Street Journal about Covid-19 vaccines relied heavily on preprints, with all but one of its scientific sources coming from an unreviewed draft paper. A reporter from the New York Times has been publishing stories in the paper discussing — and promoting on Twitter — an array of preprints, misrepresenting them so badly that Ed Yong felt compelled to call her out in a recent Atlantic article — where Yong himself pulled from some preprints, but with more care.

Despite a steep drop off in coverage of preprints after the first wave of Covid, and the media having gotten burned covering bogus preprints from charlatans and miscreants, it seems the rise of the Delta variant has also led to a rise in the preprint coverage variant.

This “second wave” of preprint coverage indicates how things could change for expert communities — most often represented in the media by journals, press officers, and the chairs of various society committees and study groups — if this trend continues, or if the repeated explorations of utilizing preprints in the media increases the comfort level among journalists and the public for such coverage.

The system of media relations that gradually became the standard over the past 50 years placed expert systems in the newsmaker position when it came to novel scientific and medical claims. If these claims could not pass muster with editors, expert reviewers, statisticians, and professional journal staff and their processes, the claims would either have to be significantly revised until they did, or would vanish in a cloud of doubt.

However, once they did pass muster and headed toward publication, vetted professional journalists would get a preview so they could arrange interviews, work with communications professionals to ensure access to authors and researchers, prepare stories and have their questions answered, and coordinate news coverage, increasing the quality and impact of the news being made.

Below is a simplified schematic of the positional aspects of this workable and proven system. Expert intermediaries include journals and all they represent; society expertise and reputational power; media officers for universities, societies, and governmental agencies; and, professional publishing and editorial personnel:

This has not been an ideal system — but name an ideal system. The standard is not “ideal” but “proven” and “reliable.” It is a proven system that has gotten better over time and can be improved further. Many of the criticisms of peer review are being fixed through diversification of editorial boards and reviewer panels, timeliness initiatives, and more.

Preprints, on the other hand, have only improved sporadically and under duress, with no broad improvements in their vetting, withdrawal, or diversity aspects. How do you know whether those reviewing preprints represent a diverse and qualified group? You don’t. The opacity is striking.

Preprints also don’t have media policies except via disclaimer. Any reporter from any outlet can cover preprints and write what they want about them. Epoch Times’ coverage is especially rank, but as noted above, the New York Times has reporters who also play fast and loose with preprints. Nobody provides them feedback from the preprint servers, which are passive and disengaged from the content they peddle.

From all this, a new, non-expert system of media coverage is emerging — one that looks worse than I expected:

In the emerging system, expert systems — the journals, the communications officers, the societies, the universities, the government agencies, and the communities of science — risk losing their newsmaking authority and positional leverage, with preprint servers becoming the news sources for science. This means that the interpreters of science become journalists and the public, and not the experts convened by journals and societies — the statisticians, expert reviewers, editors and editorial boards, and professional staff. These people become relegated to deciding which preprints they wish to refine and archive, which deserve inclusion in educational materials, and which will be cited in later reviews and evidence systems. Journals become repositories and not news sources. Societies lose their position on the information vanguard. Science and research news comes from media sources of various pedigrees, with various purposes, and with varying levels of expertise, care, and quality.

This represents a significant risk to social progress. Journals were developed as a bulwark against charlatans, half-wits, and liars. They aren’t perfect, there have been well-known failures, but across time and given the work they have to do, they’ve been very good, and society has benefited from the more organized and attenuated information system they’ve headed for the past 50-400 years. The same goes for the professional societies supporting and surrounding them, which also make errors but in general have been very good at exercising their legitimate authority carefully and to the betterment of all.

Paul Ginsparg recently claimed that a world with preprints is better, but he cherry-picked his examples, created false equivalencies, and overlooked troublesome flame-outs while ignoring the forest for a few of his favorite trees. He seemed to neglect the low-level confusion and uncertainty sown by preprints around Covid, along with explicit misinformation stirred into the mix through this funnel, and the hundreds of thousands of unnecessary deaths, rejection of vaccines, and other downsides that seem to correlate at some deep level with a lack of clear, definite information coming from trusted intermediaries and their expert systems.

Instead of a better world, we see various levels of panicky uncertainty around us as the system has been upset by preprint servers, opportunistic and/or ill-qualified journalists, and a flood of unvetted claims. Is my vaccine only 43% effective after 6 months? Or 63% effective after 9 months? Is the Delta variant 1000x more contagious for a shorter period? Or are vaccinated people safe? Are we all doomed? What should we be doing?

Why don’t we know what’s going on?

Perhaps we don’t know what’s going on because we no longer have a sense-making system based on expertise and trusted intermediaries. We have a claim-generating system, based on preprints, which fuels confusion and controversy. This is great for journalists, who can benefit from instilling uncertainty rather than centering informaation on carefully vetted facts, and for whom accountability is fleeting. And it is only possible because professional societies, journal editors, and others somehow think nothing can be done to right the ship.

This is all funded in biomedicine with money from CZI, which comes directly from stock profits from Facebook, both of which are run by one person — Mark Zuckerberg. Zuckerberg was recently described by the usually unflappable filmmaker Ken Burns as follows:

This is an enemy of the state, and I mean the United States of America. He doesn’t give a shit about us, the United States. He knows he can transcend it. He can get away to any place. And so it’s just about filthy lucre, that’s it.

When you speak with journal editors and society executives in private about preprints, there are a few themes, all based on a high but actively suppressed level of alarm. Some feel regret for having given preprint servers legitimacy through early support. Many feel too intimidated by an imagined social media backlash to speak their mind or take substantive action. The general feeling is that a system built for independence and expertise is being overtaken by techno-utopianism, self-serving funders, and pseudo-rebels.

But it’s easily fixed. Here are a few thoughts if you want to re-establish a sense-making system based on expertise, professionalism, and refinement and truth-seeking processes:

1. Change policies so that your journal no longer accepts papers posted as preprints, and get your peer journals to do the same
2. No longer allow papers to be posted on preprint servers after acceptance or publication
3. Create timeout periods around embargoed access for journalists who persist in covering preprints
4. Don’t cite preprints in committee statements or systematic reviews (except when using as examples)
5. Reassert and reinvigorate media embargoes and trust relationships with the press

Of course, the problems with preprint servers could be fixed — access could be limited to qualified people in each field; preprints could be made temporary; and, actual collaboration systems could be built to make pre-publication feedback viable. But as preprint servers are now, they are simply too messy, public, and unreliable to be placed at the headwaters of science. And science journalists have the wrong incentives, while lacking access to the right processes or expertise.

There is more than self-interest at stake in putting expert systems at the head of scientific and medical newsmaking again. Society’s information should be based on scientifically valid examination and processes, rather than on author convenience and journalistic incentives, which often involve exaggerated claims and instilling anxiety.

There is also the risk that without a layer of independent, expert, trusted intermediaries, information dissemination will be controlled by oligarchs and governments. Both are highly susceptible to corruption. There is a reason that journals have flourished in societies that are more democratic and open in general terms. China or Hungary couldn’t tolerate a large, independent scientific layer, for example.

Look at those diagrams again. Think about the importance of bringing order our of chaos, of coordinating information flows, and of the sense-making system in a time of great uncertainty. Think of the stakes — a pandemic, global warming, trust in science.

Is the emerging, chaotic, and unpredictable science newsmaking system that we’re allowing to emerge — with barely a challenge — going to help society advance on the whole?

This free post is made possible by the hundreds of paying subscribers to “The Geyser.” Thank you. If you’d like to join in supporting independent thinking and insights, please subscribe below. Or, if you already subscribe, please consider a gift subscription for a colleague. Thank you!

Subscribe now

Give a gift subscription