What is the problem?
In the last few years there have been numerous editorials and commentaries published in leading journals on scientific rigor and reproducibility. The uproar started in 2012 when Begley and colleagues wrote an editorial describing that over a 10 year period, they could not reproduce close to 90% of landmark cancer research studies [1]. While subsequent studies have brought that number closer to 50%, the fact remains that in many instances, the odds of reproducing previous peer reviewed scientific studies may be no better than flipping a coin.
In no way does this imply misconduct; the vast majority of scientific literature is published in good faith but what these studies fundamentally found was that scientists need to do a better job at “quality control”. Major findings of these studies pointed to shortcomings that included the lack of characterization and basic monitoring of many reagents: from animal models to antibodies. Of course, the histology community is familiar with the application of antibodies but many may be unaware of the sheer magnitude of the problem faced on the basic research side.
Consider that there are close to 3 million commercially available antibodies. There are over 100 suppliers with overlapping license and distribution agreements. An antibody raised against a specific protein target may be listed 10, 100 or even a 1000 times in an antibody specific search engine! The naming of antibodies also contributes to the problem. There is no systematic nomenclature for antibodies used as protein binding reagents. Often, a common name is used, but there is no convention so clone, gene, target protein or proprietary name can and are used; often interchangeably. There are hosts of analytical techniques that utilize antibodies but “validation” of an antibody in one method does not imply validity in another, further complicating antibody selection. All of this leads to confusion and when combined with insufficient documentation, reproducing experimental results is virtually impossible.
Why is this a problem?
Besides bad science? simply: time and money…..estimates place the protein binding reagent business market at 1.6 billion dollars annually. Half of that is spent in the US and half of that, close to $350 million dollars, is believed to be spent on “bad” antibodies [2]. The bottom line, the economic cost of irreproducible science could range anywhere from 10-50 billion dollars a year. To put that in perspective, the 2015 NIH fiscal budget was 30.4 billion dollars. More importantly, the cost is insignificant when one considers the potential toll on human lives of flawed research.
As the magnitude of the problem became more apparent and public exposure began to grow, the government responded…after all, taxpayer money funds the NIH and the NIH is the largest biomedical research funding organization in the world. In 2014, Frances Collins, head of the NIH, introduced new grant application guidelines that the NIH deemed important to enhance reproducibility and rigor in biomedical science [3]. Since then, grass root groups as well as formal organizations have proposed changes to the practice of biomedical science as well as the reporting of results in the literature, putting a greater emphasis on creating an environment that awards robust scientific practices [4].
Stay tuned for part two of this post, which will explore what is driving this problem, and what this problem means for the histology profession.
References
1. Begley, C.G. and L.M. Ellis, Drug development: Raise standards for preclinical cancer research. Nature, 2012. 483(7391): p. 531-3.
2. Bradbury, A. and A. Pluckthun, Reproducibility: Standardize antibodies used in research. Nature, 2015. 518(7537): p. 27-9.
3. Collins, F.S. and L.A. Tabak, Policy: NIH plans to enhance reproducibility. Nature, 2014. 505(7485): p. 612-3.
4. Freedman, L.P., I.M. Cockburn, and T.S. Simcoe, The Economics of Reproducibility in Preclinical Research. PLoS Biol, 2015. 13(6): p. e1002165.
5. Freedman, L.P., et al., The need for improved education and training in research antibody usage and validation practices. Biotechniques, 2016. 61(1): p. 16-18.
6. Van Noorden, R., Some hard numbers on science's leadership problems. Nature, 2018. 557(7705): p. 294-296.
Written by Luis Chiriboga#2019#Blog#GeneralAnatomicPathology