Monday, November 16, 2009
What's the Point of Blind Review?
A few days ago, I agreed to comment on a paper at the Pacific. Today, I was asked by a journal to review a paper with the same title as the conference paper. It occurred to me that I should decline to review it for the journal because I know the identity of the author. But then I started thinking that all I know is the name and affiliation of the author-- I don't recognize the name and don't have any particular view of the affiliation. Is simply knowing these things sufficient to undermine blind review? I suppose it does. Does this seem correct?
Subscribe to:
Post Comments (Atom)
9 comments:
Well since you do not know the person or much about the affiliation. I would say it has an effect only if you tendency is to give it a negative review.
If you knew the person and the institution are were going to give it a positive review that too would indicate a possiblity of bias. Well, unless giving it a positive review to a jerk at some overrated upset you.
The point of blind review is to reduce bias. Much like the screens that orchestral musicians play behind during auditions.
"Well, unless giving it a positive review to a jerk at some overrated upset you." should read "Well, unless giving it a positive review to a jerk at some overrated institution upset you."
Why not review it if you have no known prejudice? Your criticism should be as good as the gold you have to offer otherwise. It occurs to me that God's judgement is never really blind (by hypothesis of omniscience) but somehow is deemed just and right by the faithful in spite of that, but for no good reason other than blind trust (see Euthyphro). I have faith in you, but by a reasoned faith based on your blogging, which is more than I could say by the double-blinded faith in God's judgement that has no rational third party arbiter. Go for it.
I think it's perfectly acceptable to tell the editor you know who the author is but don't think it will bias your review and to ask if s/he is happy for you to review the paper anyway.
What puzzles me is what the point is of the author presenting at the conference if the paper is already under review and won't benefit from feedback. Presumably if s/he had responded to the points in your APA comment, those would also have been the points in your review...
If I declined to referee papers for which I knew or suspected the identity of the author, I'd never review anything... When it comes to papers that are very close to my area, I've usually come across them at conferences, or on the web, or know the author sufficiently well to recognize their interests and style.
If that's generally the case (is it? - perhaps my subdiscipline is usually incestuous), then one has to ask what the point of a process that obscures the identity of the referee but not the author is - perhaps it affords some protection to junior referees, but on the whole it just makes referees more snide, without protecting the author from bias...
Biases are often things we're not conscious of. Just because you don't feel biased doesn't mean you're not. Also, you know more than you might realize, e.g., probably the person's gender, the fact that they don't have much of a reputation and maybe other potentially biasing stuff. These facts can affect one's judgment in subtle ways. Perhaps fear of being biased as a result of such facts can be outweighed by considerations like those offered by anon 6:55am, but I think they should be taken seriously.
I thought one of the main purposes of blind review was to put the big shots and the unknowns on something of a level playing field---so that individuals with a reputation can't just publish whatever garbage they like, and individuals just starting out don't have a really difficult time publishing their research.
If that's right then it seems that you shouldn't review an article when you know the person's name (the fact that you know that this person isn't a big shot is going to make it easier for you to give it a negative review).
Sort of off topic, but when do you pull a paper?
Initial submission took a year and change before I received R&R. Resubmitted. I think it took about 6 months to hear back. One ref said 'Yes', one ref said 'Wait, I've thought of a new objection'. Ref says that this objection is basically the thing that stands in the way of his recommending acceptance. Objection had been addressed in initial submission in section cut to satisfy the ref's desire to have paper streamlined. Resubmit. I've waited 5 months now and no word. Upon resubmitting, one of the refs (I believe, it would be a crazy coincidence if it wasn't the ref) used the google to trace paper back to my webpage where the title was listed as a conference presentation. I can't seem to get the journal to do much to get this guy to act and I suspect that I've rubbed the ref the wrong way by pointing out (delicately and politely) that the ref's new objection was just the old objection addressed back in 2007 in a section that ref said to cut. So today I floated the possibility of pulling the thing if I didn't receive some sort of up/down decision within the month. Should I pull if I don't hear something by mid-December? Help. I need advice. When do people give up on a journal and start over?
If you think you know the author, you must inform the editor. Funny thing though, in each of the handful of such cases I've had, the editor never once even cocked an eyebrow let alone assigned the paper to someone else.
Hmm, maybe next time I'll try "Not only do I know the identity of the author, but also, we totally did it."
Post a Comment