Just a thought on peer reviewing - I can’t help myself.

Jeff Leek

Today I was thinking about reviewing, probably because I was handling a couple of papers as AE and doing tasks associated with reviewing several other papers. I know that this is idle thinking, but suppose peer review was just a drop down ranking with these 6 questions.

  1. How close is this paper to your area of expertise?
  2. Does the paper appear to be technically right?
  3. Does the paper use appropriate statistics/computing?
  4. Is the paper interesting to people in your area?
  5. Is the paper interesting to a broad audience?
  6. Are the appropriate data and code available?

Each question would be rated on a 1-5 star scale. 1 stars = completely inadequate, 3 stars = acceptable, 5 stars = excellent. There would be an optional comments box that would only be used for major/interesting thoughts and anything that got above 3 stars for questions 2, 3, and 6 was published. Incidentally, you could do this for free on Github if the papers were written in markdown, that would reduce the substantial costs of open-access publishing.

No doubt peer review would happen faster this way. I was wondering, would it be any worse?