Adam Mastroianni has a thoughtful and interesting article on peer review. The idea of having outside “experts” evaluate a prospective paper before it’s published in a journal seems to be obviously the right thing and something that has been the norm in scientific—and then all academic research—practically from the beginning.
Mastroianni begs to differ. In the first place, peer review, far from being the ancient procedure that we imagine is actually quite new. Before 1960, it was rare and none of Einstein’s papers, for instance, were peer reviewed. In fact when a journal decided to peer review one of his papers, he was so surprised and upset that he withdrew the paper and published it elsewhere.
Mastroianni’s second point is more telling. He describes peer review as a massive experiment that was flawed from the beginning and has, in fact, failed. He claims that not only is there no evidence that peer review improves science but there is plenty to suggest that it’s actually made it worse. He notes—correctly I think—that peer review makes sense only if believe science is about preventing bad ideas. But, he says, it’s not. It’s about finding the best ideas, something that peer review will never do because its raison d’être is to find and suppress bad ideas.
The subject of peer review probably seems of interest only to academics but, in fact, it affects us all. Despite this so called safeguard, about three quarters of papers in critical areas such as cancer research fail to replicate and many are outright fraudulent. That means that millions—or even billions—of dollars are being misappropriated in areas where we all have a vital interest.
I believe Mastroianni is correct but I don’t hold out much hope for the end of peer review until the university system itself finally collapses. Of course, many believe that process is already well underway so who knows?