Survey Results — Top Level

Last week’s “sentiment survey” — my effort to see how people are feeling about some of the main controversies, initiatives, directions, and beliefs in scholarly publishing — garnered a 15% response rate (249 responses). Thanks to everyone who responded.

The survey instrument consisted of a set of sliders, allowing respondents to indicate their level of agreement with each statement.

• For instance, if I stated, “The Beatles are overrated,” you could use a slider going from “Strongly Disagree” to “Strongly Agree” to indicate the degree to which you think the Beatles are or are not overrated, with the midpoint marked as “Neither Agree nor Disagree.”

Whether this survey is “valid” is a question I’ll leave for others. It’s informative, it’s as far as I know unique, and audience surveys that ask new questions are useful because they provide a starting point for further exploration — or they provide a check on misconceptions run amok. This survey is that kind of instrument — one that asks questions and gauges responses in a way others have not, as far as I know.

Some may argue there is the potential for selection bias here. After all, if you read “The Geyser,” chances are you are a skeptic of utopian views, or you’ve been gradually convinced some ideas fall short, or both. However, with more readers here than there are members in most organizations in this space, it’s not a small representation of the industry, and the selection bias is probably less than if such organizations ran a similar survey. Readers of “The Geyser” come from all over, work in many different roles, span all career stages, and hail from many working domains.

Now that we’ve dispensed with some preliminaries, here are the top-level results showing the degree of agreement with each statement. The midpoint is in red.

The Left Side of the Midpoint

The following are my interpretations of the items to the left of the midpoint — where there is not strong agreement with the statements. Based on long experience with the issues and various attitudes that would also lie to the left of the midpoint, I’ve elaborated a little for each.

• Trust in AI companies is low. This may be because we’re in the early days of LLMs and such tools, or because AI hype makes people wary, or because the litany of bad examples keeps coming and seems to have a momentum — or, all of the above, and more.
• OA is not viewed as having improved things. Given the amount of cheerleading out there, I’d have expected this level of agreement to be higher. However, for those who pay attention to surveys of authors, listen to editors and researchers, and have remained open to the idea that OA might have inherent problems, this isn’t surprising.
• Funder policies are unwelcome and unseemly. I think there is more discomfort around funder behavior than anyone has said openly.
  Funders are insulated by their power and money, are using authors as leverage to gain control of publishing policy, and maybe we’re afraid to make them mad.
• Preprints did not help during the pandemic. Again, cheerleading at the NLM and in other venues aside, most level-headed people know intuitively — and facts have borne this out — that servers run with such a light touch are going to have problems.
• There isn’t much enthusiasm for where we’re headed. Combining three questions here — about optimism, leadership, and our current direction — it’s pretty clear people think we’re not doing very well, need to change direction, but may not have the leadership needed.

The Right Side of the Midpoint

The following are my interpretations of the items to the right of the midpoint — where there is stronger agreement with the statements. Based on long experience with the issues and various attitudes that would also lie to the right of the midpoint, I’ve elaborated a little for each.

• Preprint servers aren’t fit for purpose. While there aren’t specifics here, there seems to be a strong feeling that preprint servers need to change, and aren’t up to whatever task they have been assigned. The mismatch between their stated intent and how they’re actually being used and abused may account for some of this.
• Publishers — just a little better than “meh.” Coming in with a rousing C+, publishers have acquitted themselves just well enough to get a passing grade over the past 10 years, at least based on these results.
• Gatekeepers have strong support. In one of two strong sources of agreement, respondents felt gatekeepers who are expert, dispassionate, and professional remain necessary.
• Peer-review and editorial review help. In the strongest source of agreement, peer-review and editorial review are seen as helpful for authors and papers.
  • Spoiler — researchers really agree with this.
• We like our jobs well enough. While still in the C+ zone, there’s at least some comfort with our kids taking jobs in scholarly publishing. I think this would have registered at 80+ just two decades ago. If so, the reputation, remuneration, and prestige involved may be diminishing assets.

Inverting the Graph

For the sake of visual learners, here are the same results, but inverted so that every question is a negative (e.g., “The Beatles are not overrated.”). This might help by throwing a slightly different light on some things by showing where this is strong agreement with the negative versions of the same statements.

I can imagine some caterwauling about this survey just showing how academics, publishers, and others are resistant to change. I don’t think that’s a good-faith interpretation by someone intent on understanding serious people doing serious work in a serious manner.

• If you try to change a field at the existential level — if you wanted lawyers to give up their role as officers of the court, or basketball players to stop playing by the rules that define the game — you can’t expect much to change.

Thanks again to everyone who took the survey. A subscriber-only breakdown comparing responses from the four most interesting groups — librarians, faculty/researchers, consultants, and editorial — will be sent out tomorrow. One group is noticeably out of sync with the others . . .