6533b7d1fe1ef96bd125c2b3
RESEARCH PRODUCT
A simulation of disagreement for control of rational cheating in peer review
Mario PaolucciFrancisco Grimaldosubject
Artificial social systems peer review agent-based simulation trust reliability and reputationComputer scienceMechanism (biology)business.industryCheatingmedia_common.quotation_subjectControl (management)Mechanism basedArtificial social systemsComputer securitycomputer.software_genreCrowdsourcingData sciencePeer reviewWorld Wide Webtrust reliability and reputationControl and Systems EngineeringHomogeneouspeer reviewQuality (business)businessPsychologycomputeragent-based simulationmedia_commondescription
Understanding the peer review process could help research and shed light on the mechanisms that underlie crowdsourcing. In this paper, we present an agent-based model of peer review built on three entities - the paper, the scientist and the conference. The system is implemented on a BDI platform (Jason) that allows to define a rich model of scoring, evaluating and selecting papers for conferences. Then, we propose a programme committee update mechanism based on disagreement control that is able to remove reviewers applying a strategy aimed to prevent papers better than their own to be accepted (rational cheating). We analyze a homogeneous scenario, where all conferences aim to the same level of quality, and a heterogeneous scenario, in which conferences request different qualities, showing how this affects the update mechanism proposed. We also present a first step toward an empirical validation of our model that compares the amount of disagreements found in real conferences with that obtained in our simulations. © 2013 World Scientific Publishing Company.
| year | journal | country | edition | language |
|---|---|---|---|---|
| 2013-01-01 |