WOW> I just read a sobering and somber OpEd in SLATE: Cancer Research is Broken. Well, it’s not surprising to anyone in the biz, including those of us in bioinformatics and specifically us working at A2IDEA. Heck, we re-analyze bit and pieces of public data for other researchers ALL.YEAR.LONG. While, we’ve never tried to completely ‘reproduce’ or ‘replicate’ a seminal paper, we’ve seen first hand some data quality concerns that would prevent us from putting forth conclusions from hence data.
If you don't’ have time to read it.. Here’s some highlights, factual, not the opinion part. Although, the opinion and subsequent commentary are interesting.
When Cancer research results are tested in a private lab:
A team from Amgen, over ten years, showed only 6 of 53 “landmark” paper results were reproduced.
A team from Bayer found only 20-25 % results were ‘completely in-line’ with original publications.
These findings are so well accepted that venture capitalist know, at most, 50% of academic published results will actually be reproduced in an industrial laboratory.
An international effort, Reproducibility Project for Cancer Biology, was started in 2013 to diagnose and provide a report of findings. With an enormous budget, 2 Million, and a goal of only reproducing 50 studies, they’ve only reported their delays. UG!
They have recently published a plan for the reanalysis of “COT drives resistance to RAF inhibition through MAPK pathway reactivation” by Johannessen and colleagues, published in Nature in 2010 (Johannessen et al., 2010) and some insight into how to curate papers to identify specific materials and detailed methods. They came up with an Identifiability Issue, which is akin to Gene Ontology for reagents and methods.
Good plans, I believe. Let’s see if they gain traction and stick.