By Porter Anderson, Editor-in-Chief | @Porter_Anderson

‘The Integrity of Published Academic Work’

The UK’s Birkbeck, University of London has announced itself the recipient of a US$99,000 grant from the Andrew W. Mellon Foundation to fund an investigation of the peer review process. Birkbeck is a part of the public research university that specializes in evening study.

This is the second grant Birkbeck has received from the Mellon Foundation, both under the heading of scholarly communications. In 2015, the university received US$741,000 for its Open Library in the Humanities project.

This time, the grant’s focus on “investigations of methods of assessing the nature and quality of contributions to scholarship,” according to Mellon listings, and is designed, university officials say, “to determine the validity, quality and originality of research for publication.”

At issue here are “aims to uphold the integrity of published academic work and to ensure poor quality or inaccurate works do not reach publication,” in the description offered to the media in press materials.

The research team leader on the project is Martin Eve, a professor of literature, technology, and publishing. In a prepared statement, Eve is quoted, saying, “Every academic in the world is familiar with peer review.

Martin Eve

“We all participate in this system. However, surprisingly little is known about what peer review looks like at scale.

“Using a set of cross-disciplinary methodologies, we’ll study the writing patterns of reviewers and look for trends that can give us insight into how reviewers respond to requests for evaluation.”

The university’s title for the project is”Reading Peer Review,” and, per press materials, “will analyze the role of peer review in the research cycle to develop better ways of using expert opinion to assess and improve the literature available.

“It will investigate the peer review database at PLOS ONE, the largest scientific journal in the world. The reports, which will be strictly anonymized and treated as confidential, will be examined using a range of close-reading, distant-reading, and stylometric approaches.”

Speaking for PLOS (Public Library of Science), Veronique Kiermer, its executive editor, is quoted, saying:

Veronique Kiermer

“Working with the Birkbeck group allowed us to find ways to leverage our large dataset of reviews while maintaining confidentiality,

“This is a great opportunity for PLOS to partner with experts to gain insights and promote evidence-based approaches to peer review.”

The challenge of this project, of course, has to do with what Birkbeck refers to as the “oft-hidden process” of peer review.

Joining Eve and Kiermer on the project are:

• Cameron Neylon, Curtin University, Australia;
• Daniel O’Donnell, University of Lethbridge;
• Jennifer Lin, director of Product Management at Crossref;
• Damian Pattinson, vice-president of publishing innovation at Research Square; and
• Samuel Moore, King’s College London.