Peer reviewers equally critique theory, method, and writing, with limited effect on the final content of accepted manuscripts
Bibliographic Data
| ID | 21443070 |
|---|---|
| Authors | Dimity Stephen (0000-0002-7787-6081, German Centre for Higher Education Research and Science Studies, corresponding author) |
| Year | 2022 |
| Volume | 127 |
| Issue | 6 |
| Pages | 3413-3435 |
| Publication date | 2022-06-01 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Scientometrics (JOURNAL) |
| Journal identifiers | ISSN: 0138-9130 • E-ISSN: 1588-2861 |
| Publisher | Springer Science and Business Media LLC (PUBLISHER) |
| DOI | 10.1007/s11192-022-04357-y |
| PMID | 35431366 |
| OpenAlex | W4224254650 |
| Language | EN |
| Citations received | 1 |
| References cited | 29 |
The primary aims of peer review are to detect flaws and deficiencies in the design and interpretation of studies, and ensure the clarity and quality of their presentation. However, it has been questioned whether peer review fulfils this function. Studies have highlighted a stronger focus of reviewers on critiquing methodological aspects of studies and the quality of writing in biomedical sciences, with less focus on theoretical grounding. In contrast, reviewers in the social sciences appear more concerned with theoretical underpinnings. These studies also found the effect of peer review on manuscripts’ content to be variable, but generally modest and positive. I qualitatively analysed 1430 peer reviewers’ comments for a sample of 40 social science preprint-publication pairs to identify the key foci of reviewers’ comments. I then quantified the effect of peer review on manuscripts by examining differences between the preprint and published versions using the normalised Levenshtein distance, cosine similarity, and word count ratios for titles, abstracts, document sections and full-texts. I also examined changes in references used between versions and linked changes to reviewers’ comments. Reviewers’ comments were nearly equally split between issues of methodology (30.7%), theory (30.0%), and writing quality (29.2%). Titles, abstracts, and the semantic content of documents remained similar, although publications were typically longer than preprints. Two-thirds of citations were unchanged, 20.9% were added during review and 13.1% were removed. These findings indicate reviewers equally attended to the theoretical and methodological details and communication style of manuscripts, although the effect on quantitative measures of the manuscripts was limited
CLARITY · Epistemology · Interpretation (philosophy) · Peer review · Preprint · Presentation (obstetrics) · Quality (philosophy) · Redaction · World Wide Web · Academic Publishing and Open Access · Academic Writing and Publishing · Computer Science · Medicine · Psychology · scientometrics and bibliometrics research
Ggplot2
The evolving role of preprints in the dissemination of Covid-19 research and their impact on the science communication landscape
Welcome to the Tidyverse
Talking Past Each Other
Quantitative Analysis of Qualitative Information From Interviews
Frame Search and Re-search
The Manuscript Review and Decision-Making Process
Revising as Reframing
| Citations per year | 1 |
|---|---|
| Citation span | 2025 - 2025 (1) |
| Highly cited | No |