Center for Analysis and Design of Intelligent Agents

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== Reviewing Scientific Papers =====

===Concepts===

| Conference  | A gathering of scientists for the purpose of presenting their own work to each other.  |
| Workshop    | Conferences often have workshops on selected topics. These draw crowds from 8-40 people, depending on the size of the conference. Typically papers submitted to workshops are published in the conference proceedings.  | 
| Conference Proceedings   | A publication of all the work presented at the conference, in the form of scientific papers.  |
| Journal     | A professional outlet/publication for scientific work.   |
| Technical report   | Anything can be published a technical report. Tech reports published by university departments are typically not peer-reviewed.  |
| Measuring scientific prestige  | Scientists compete. They compete for **prestige**, as measured by: \\ 1. Number of publications. \\ 2. Number of publications they are listed as first author on. \\ 3. Number of publications per year. \\ 4. Number of quotations by others to their work. \\ 5. Quality of publications that papers get published in.  |
| Prestige of scientific outlets  | From low to high: \\ * Tech report \\ * Workshop paper \\ * Conference paper \\ * Book authored \\ * Journal paper  |

\\
\\
\\
\\

===Peer Review===

| What is it?  | Before replication of results can be undertaken by the scientific community, results must be published. When a scientist reviews another scientist's current work, it's called "peer review".   | 
| The peer  | A scientist should be an authority in his/her field -- is there anyone who has a higher authority? Yes, the scientific method, in other words the **scientific** community. To review their work current work scientists enlist the practical embodiment of this community -- their peers.  |
| How current scientific work gets evaluated   | Via replication of results -- but first results must be published. Deciding what gets published, and how, is the role of the peer review process.    |
|   |   |

\\
\\
\\
\\

===The Peer Review Process===

| Step 1  | Scientist does research, writes up results and submits a scientific paper to a selected outlet.   |
| Step 2  | Editor receives submission, decides who should review. The selected review group, typically 3 or more scientists knowledgeable in the field in question, is called the peer review group.   |
| Step 3  | Editor sends paper to peer review group with a deadline for returning their review, plus instructions.  |
| Step 4  | Editor gets reviews from reviewers.  |
| Step 5  | Editor has to decide, based on reviews, whether to (1) accept paper as-is, with no changes (very rare!); (2) accept paper with minor revisions; (3) accept paper with major revisions; (4) reject paper.  |

\\
\\
\\
\\

===Peer Review Instructions===

| Several categories are used when reviewing  | Overall quality of the work; Novelty/Significance of the contributions to the field in question; Clarity of the writing; Language/writing quality; Adherence to guidelines (paper length, abstract length, etc.) for the publication in question.   |
| Quality of Work       | Use your experience with the subject, and of course with other papers. Look at the content, not where the author comes from or where he/she does the work. Be honest. Be fair. If it's bad, say so. You do not do anyone a favor by trying to "be nice" -- in fact, being "nice" in this context translates to being cruel to all the readers, as well as the author (who will then not know that the quality is not good enough!). **Always give reasons and examples to back up your comments.** |
| Novelty/Significance  | Sometimes papers are written that hardly make a dent in propelling science forward; sometimes papers are breakthrough. Here it is important to think about examples of work you know very well, and try to place this work in that context, to see where it fits in impact.  |
| Clarity of writing    | This is a prerequisite to getting a paper published. It doesn't matter how revolutionary the work is, if it's not communicated properly nobody will read it.   |
| Language quality      | Nothing is as annoying as a good paper that falls flat on bad use of English (which, let's face it, is the language of science, at least until China overpowers us with Mandarin). Be brutal! Do not be nice to badly written papers. NB: There is always room for improvement in this regard.   | 

\\
\\
\\
\\

===Being a Peer Reviewer===

| Rule number one  | When you are requested to be a peer reviewer, accept. Firstly, it is your duty as a scientist to help out, even if it is not paid work! Secondly, it can really improve your own writing to read other people's papers!   |
| When you get the paper + instructions  | Note the deadline, then block out 1/2 day well before the deadline to do the review.  |
| Follow the instructions  | If you don't get any instructions, use the above that I gave you.   |
| First mistake   | Thinking it's "really easy". Sometimes a paper is obviously flawed -- but that is not the most common case. Most of the time it requires a lot of work, careful analysis and careful reading to see whether a paper is worthy of publication. \\ Nevertheless, the bottom line is always that the author is ultimately responsible.   |
| Second mistake  | Thinking it's not so important. Your review could affect a fellow scientist's career!   |

\\
\\
\\
\\


EOF