[HTML][HTML] Making the scientific literature fail-safe

A Casadevall, FC Fang - The Journal of Clinical …, 2018 - Am Soc Clin Investig
The Journal of Clinical Investigation, 2018Am Soc Clin Investig
Critical systems are designed to be failsafe. This does not mean that failures cannot occur,
but rather that redundant and compensatory mechanisms are engineered into the system to
detect and mitigate failures when they occur. The scientific literature is the critical system by
which scientific findings are communicated and archived for subsequent reference and
analysis. Hence, the reliability of the scientific literature is of the utmost importance to
society. However, in recent years, rising numbers of retracted articles, reproducibility …
Critical systems are designed to be failsafe. This does not mean that failures cannot occur, but rather that redundant and compensatory mechanisms are engineered into the system to detect and mitigate failures when they occur. The scientific literature is the critical system by which scientific findings are communicated and archived for subsequent reference and analysis. Hence, the reliability of the scientific literature is of the utmost importance to society. However, in recent years, rising numbers of retracted articles, reproducibility problems, and inappropriately duplicated images have increased concern that the scientific literature is unreliable (2–5). Contributing factors may include competition, sloppiness, prioritization of impact over rigor, poor experimental design, inappropriate statistical analysis, and lax ethical standards (6, 7). Although the number of questionable publications represents a very small percentage of the total literature, even a few problematic publications can reduce the credibility of science. Hence, it is important to redouble efforts to improve the reliability of scientific publications. We suggest a seven-point approach to reengineering the scientific literature so that it is better able to prevent and correct its failures. i. Improving graduate and postgraduate training. Training is the foundation of all scientific endeavors. Contemporary graduate scientific training is designed to prepare trainees to perform deep investigation into a highly specialized area (8) but does not necessarily provide students with a broad scientific background. Students are taught in a guild-like environment by a mentor who may or may not have been trained in good scientific practices. Consequently, there is no guarantee that programs are consistently producing scientists who are adequately prepared to do good science. Improving postgraduate training to ensure that trainees are well versed in scientific rigor, statistical analysis, experimental design, and ethics can improve the quality of the scientific literature by improving the quality of the research itself. ii. Reducing errors in manuscript preparation. Roughly one of every 25 articles in the biomedical literature contains an inappropriately duplicated image (4). The majority of inappropriate image duplication results from simple errors in figure assembly (9). However, a minority of these represents intentional efforts to mislead the reader, which constitutes scientific misconduct. Involving multiple individuals in figure preparation prior to manuscript submission may reduce the likelihood of error and also discourage intentional deception. iii. Presubmission criticism. Although peer review is intended to detect and correct errors prior to publication, the process involves only a small number of reviewers and is well known to be imperfect (10). Critical input from a broader range of colleagues may lead to identification of weaknesses in a manuscript and allow authors to improve the quality of their published work. Presubmission criticism may be informally obtained by asking others to read a manuscript before submission or by posting the manuscript on a preprint server and alerting colleagues in the field that the data are available in prepublication form. Both authors of this commentary have received presubmission criticism of manuscripts posted as preprints that led to improvements. A more longstanding mechanism for obtaining presubmission criticism is to present unpublished data at meetings and seminars. iv. Robust review and editorial procedures. After a manuscript is submitted for publication, the peer review and editorial processes are major checkpoints for quality improvement. Reviewers can identify errors, and training may improve …
The Journal of Clinical Investigation