Will the Next Generation Fix It?

I recently got into an exchange about studies purporting to show little difference between published works and their corresponding preprints. One such analysis came out of the recent Peer Review Congress.  

Knowing this underwhelming area of the journalology literature well, I asked the promoters of this analysis whether certain obvious and well-known confounders were accounted for in the studies — confounders like the propensity for authors to deposit preprints after peer-review has occurred and acceptance is either a certainty or publication has already occurred, as well as the fact that most papers are rejected after multiple rounds of peer-review across a set of field-adjacent journals, with each review helping improve the paper, especially the first one or two.

This led to an exchange in which one author kept claiming that various studies had shown various things, and me asking again if other well-known confounders were also covered and accounted for in any or all of them. After a time, another participant in the conversation veered from evidence and began accusing me of letting my idiosyncratic experience drive my views.

I responded to this rhetoric to ask if this person could show me one study — just one — that accounted for the major confounders I’d elucidated and still found a lack of meaningful difference between preprint and published work. If there is such a paper, after all, I’d love to see it.

The silence that followed was deafening.

This is an all-too-common occurrence. When challenged to produce more than comfortable talking points and easy “open” rhetoric, the reality of the world often becomes oppressive and unfriendly for advocates. Various claims about the benefits of OA and its ilk have evaporated under such scrutiny every time, as far as I’m aware.

Yet, transgressions of scholarly publishing norms are viewed by some as stylish, even if they are corrosive and ultimately fail to amount to anything more. In a recent interview, Cliff Lynch of CNI said he believes the peer review system is “slowly collapsing,” and expressed concern that experimental peer review approaches launched to deal with the explosion of biomedical preprinting during the peak of the Covid pandemic won’t endure or prove viable in the long-term.

If the system is collapsing, it’s largely because termite-like transgressive ideas have chewed away at the underpinnings — from preprint servers to predatory publishers to mega-journals to the asymmetrical supply-demand deal with Chinese scientific institutions. Editors are discouraged, and volunteer reviewers feel that their contributions are trivial. This isn’t a great way to support either important group, especially because their purported ineffectiveness has never been effectively demonstrated, as noted above. In fact, they generally do a ton of fantastic work for authors and readers alike, and should be recognized for it.

The pressures on the peer-review system have been well-know for years, but many responses have only exacerbated the problems, fomented unnecessary skepticism and cynicism, and fostered business models that allow interlopers to have inexplicable influence.

A lot of this seems to be caused by people in their middle-age and older attempting to appear young and dangerous by embracing techno-utopian ideas, many of which are themselves outdated. This particular model of radicalism on multi-vitamins has itself aged out, even if those espousing it don’t realize it . . . dude.

The effects are being felt by actual young people, as a recent survey from Poynter, MediaWise, and YouGov shows — older people still have strong trust in media, while distrust increases as age drops. That is, people who have grown up closer to the epicenter of the misinformation age have registered that something is amiss, and their level of trust has dropped commensurately.

This is bad news, but something I and others have been worried about for years. As publishers abdicated their role as trusted intermediaries — at precisely the juncture in the history of information when it made the most sense and was most urgently needed — trust in science and truthful information has faltered. An overview of common techniques that have flourished to push fake information in the absence of trusted intermediaries and expert gatekeepers was recently expounded in the British press as reporters elaborated on the survey’s findings.

Put it all together, and I find myself revisiting a quote that remains relevant:

. . . our class has not delivered for the people outside it. On our watch, government and other public institutions have deteriorated. Part of the problem is that, steeped in an outsider, pseudo-rebel ethos, we never accepted the fact that we were a leadership class, never took on the institutional responsibilities that go with that acceptance, never got to know or work with people not in our class, and never earned the legitimacy and trust that is required if any group is going to effectively lead.
— David Brooks, The Atlantic

A generation of aging, pseudo-rebel transgressives is consciously eroding the basis of scientific communications — expert review and expert, trusted intermediation. They’re pushing inadequate studies and inflated and corrosive claims on Twitter. They’re expounding perspectives that don’t stand up to scrutiny, and then complaining that those scrutinizing them are not showing their views the proper respect — as if deference is more respectful than engagement and critique. We’ve allowed the effects of the past 20 years of transgressions to become so obvious that society has noticed, and distrust in what we distribute is increasing the most among young people.

Are we going to keep peddling half-baked “studies” trying to show our transgressive bona fides? Are we going to spin studies against the evidence they show in order to back up our favored pseudo-rebel rhetoric?

Transgressives keep chewing at the foundations of scholarly publishing with tendentious rhetoric and inadequate evidence.

Society is seeing the holes and damage.

Are we just leaving another mess for our children and grandchildren to clean up?