I’ve often written that reproducible research is the best way to do science. Over at Molecular Brain, it’s editor, Tsuyoshi Miyakawa, has a long and interesting editorial that argues that if you aren’t providing raw data, you aren’t doing science. In other words, reproducible research isn’t the best way of doing science, it’s the only way.
The editorial, No raw data, no science: another possible source of the reproducibility crisis, relates some startling facts. For example, in cancer research the reproducible rate, according to one study is only 20–25% and according to another it is only 11%. In cancer research. This is serious research with life altering consequences but more than 75% of the time the results can’t be verified.
It gets worse. Miyakawa’s policy is that if the results appear “too beautiful to be true” he will asks the authors for raw data before sending the paper out for review. That’s an excellent policy but what happens next is also startling. Since 2017, Miyakawa has asked for raw data 41 times. In 21 cases the papers were withdrawn without providing the data. Of the other 20 cases, only one paper was accepted. Miyakawa rejected the others for providing insufficient data.
Even if you haven’t achieved Irreal levels of cynicism, it’s hard to avoid concluding that in some of these cases fraud is involved. Miyakawa concludes the same. Journals could make significant inroads into this problem if, like Molecular Brain, they insisted that researchers provide their data. Organizations providing research grants could do their part by insisting that the raw data gathered by the research they fund be made available. What’s happening now is absurd.