6533b7d4fe1ef96bd1262795
RESEARCH PRODUCT
false
subject
Index (economics)Information retrievalComputer scienceMetaphormedia_common.quotation_subject05 social sciencesLibrary and Information Sciences050905 science studiesComputer Science ApplicationsTest (assessment)Sample size determinationLearning to rank0509 other social sciences050904 information & library sciencesmedia_commondescription
Abstract This paper presents an index that measures reviewer contribution to editorial processes of scholarly journals. Following a metaphor of ranking algorithms in sports tournaments, we created an index that considers reviewers on different context-specific dimensions, i.e., report delivery time, the length of the report and the alignment of recommendations to editorial decisions. To test the index, we used a dataset of peer review in a multi-disciplinary journal, including 544 reviewers on 606 submissions in six years. Although limited by sample size, the test showed that the index identifies outstanding contributors and weak performing reviewers efficiently. Our index is flexible, contemplates extensions and could be incorporated into available scholarly journal management tools. It can assist editors in rewarding high performing reviewers and managing editorial turnover.
| year | journal | country | edition | language |
|---|---|---|---|---|
| 2019-02-01 | Journal of Informetrics |