Google Scholar and “Fishy” GPT
A study of Google Scholar shows yet again how easily exploitable this poor technology implementation has become
A recent paper showing how LLM-generated papers have infiltrated Google Scholar has a humdinger of a paragraph in its middle:
Google Scholar’s central position in the publicly accessible scholarly communication infrastructure, as well as its lack of standards, transparency, and accountability in terms of inclusion criteria, has potentially serious implications for public trust in science. This is likely to exacerbate the already-known potential to exploit Google Scholar for evidence hacking . . . and will have implications for any attempts to retract or remove fraudulent papers from their original publication venues. Any solution must consider the entirety of the research infrastructure for scholarly communication and the interplay of different actors, interests, and incentives.
What the authors fail to note is that Google Scholar is an understaffed, underfunded, and idiosyncratic technology sideshow at Google. But they get the main points right.
An interesting twist in the study is how fish farms and aquatic plants pop out of their analysis as concepts fed into Scholar via fake GPT-generated papers. It’s not clear why this is the case.
Another paragraph worth quoting from the paper:
. . . GPT-fabricated, questionable papers are seeping into most parts of the online research infrastructure for scholarly communication. Platforms on which identified papers have appeared include ResearchGate, ORCiD, Journal of Population Therapeutics and Clinical Pharmacology (JPTCP), Easychair, Frontiers, the Institute of Electrical and Electronics Engineer (IEEE), and X/Twitter. Thus, even if they are retracted from their original source, it will prove very difficult to track, remove, or even just mark them up on other platforms. Moreover, unless regulated, Google Scholar will enable their continued and most likely unlabeled discoverability.
Good reading, and recommended.