Scientific research is founded on a principle of reproducibility. No matter who is conducting an experiment, it must yield the same outcome if the result is to be trusted. Recent years, however, have seen a number of high-profile failings in scientific reproducibility.
The Open Science Collaboration, an organization consisting of psychologists dedicated to defining reproducibility in current psychology research, conducted one of the largest coordinated efforts to survey reproducibility to-date. In 2015, the group published a report in which members attempted to replicate the findings of 100 papers published in 2008 from high-ranking psychology journals. The conclusion – only 40% of previously reported results could be replicated. Among a number of factors, the authors list the pressures of the academic system as potential motivators of irreproducible work. Scientists need to publish novel results to secure promotions, and this pressure may push exaggerated and unreliable results.
Biologists may look at the Open Science Collaboration report and assume that these sort of reproducibility issues are a failing unique to psychological science – people are complex, and studying human behavior might just be harder to control than biological systems. Unfortunately, however, biology is not immune to the problem of irreproducible data. Just a few days ago, a story broke of suspected fraudulent data behind one of the leading theories of Alzheimer’s disease. Sixteen years ago, the now-famous neuroscientist Sylvain Lesné published an article in 2006 in Nature describing evidence for what is known as the “amyloid hypothesis”. This theory claims that Alzheimer’s disease is caused by the buildup of sticky amyloid beta protein plaques that clog up the brain. In this work, Lesné and his colleagues described a subtype of amyloid beta and provided evidence to suggest it caused dementia in a rat model of neurodegeneration. That work proved seminal to the Alzheimer’s field, and would go on to encourage hundreds of follow-up studies into the amyloid biology as a driver of Alzheimer’s, and the search for plaque-clearing therapies.
While this research led many in the Alzheimer’s field to study amyloids, a few scientists quietly raised concerns over the validity of the data during the past decade. Recently, these concerns have become much harder to ignore. Cassava Sciences, a clinical stage biotech focused on neuroscience-based therapies, claimed that their experimental drug Simufilam improved Alzheimer’s symptoms by clearing amyloid plaques. Two unnamed neuroscientists were extremely skeptical of these claims. So skeptical, in fact, that they shorted Cassava Sciences’ stock, and assembled a legal team to file a petition detailing their concerns to the FDA. While the claims are still under review at the FDA, this filing has invited increased scrutiny of Lesné’s papers, with leading scientists raising serious doubts about dozens of his papers.
While many are focused on how these fraudulent data have set back Alzheimer’s research, there is perhaps an interesting lesson in the way in which this story has come to light. Mainly, why now? What changed, since the initial publication sixteen years ago, that made the concerns over fraudulent data so public? The answer, in part, is that academic science has been launched into the biotech sector. Under the private space, the same work is viewed with new scrutiny with higher stakes. Those studies that used to be focused on rodent brains are now motivating a drug for human patients. Real human lives are becoming involved in the amyloid hypothesis, and real money is at stake depending on the success or failure of the therapy.
Biotech takes academic work and attempts to apply it. The application of science in the biotech space presents a unique stress-test to biological theory – a level of pressure that may be slow to grow in a purely academic setting. The story of Lesné’s work and Cassava Science’s drug illustrates this perfectly; a dubious theory persisted in academia despite quiet concerns, but quickly received a storm of criticism once biotech application began. As the biotech sector matures, the iterations of basic science discovery to biotech implementation will quicken. As this feedback loop accelerates, biotech is likely to serve a role in fact-checking academic theory, adding another layer working to ensure scientific reproducibility.