The COVID-19 pandemic has triggered the biggest mobilization of scientific effort in a generation. Scientists from fields as diverse as immunology and computer science quickly pivoted to studying drivers of the epidemic and potential countermeasures. More than 54,000 articles relating to the SARS-CoV-2 virus have been published in academic journals in the biomedical and life sciences to date.
This scientific surge is astounding and inspiring, but it has produced some ethical dilemmas. The urgency of the crisis has led to a proliferation of studies, some of which short-circuit the most rigorous scientific standards. Results often get disseminated to the public before they’ve been reviewed by experts, which can lead to a situation in which doctors, politicians and others advocate unproven cures.
BALANCING SPEED AND RIGOR
The gold standard for scientific learning is the randomized, controlled trial (RCT), in which a group of participants is randomly assigned to receive either the treatment being investigated or a comparison treatment, which might be a placebo. Randomly assigning participants to these two groups ensures that the groups are similar, reducing the possibility of bias.
Although SARS-CoV-2 vaccines are being tested using RCTs, nonrandomized studies have been common for COVID-19 therapies. Many have occurred within “expanded access programs,” through which the Food and Drug Administration allows patients to access therapies that have not yet received marketing approval. For instance, for convalescent plasma (blood plasma from disease survivors), evidence from case reports and animal studies was provocative enough to justify launching an RCT to test it for COVID-19. However, the federal government instead approved an expanded access program through which physicians administered plasma to nearly 70,000 patients without any control groups. Without randomization, researchers have not conclusively shown that improved outcomes are the result of taking plasma. Nevertheless, the FDA granted an emergency-use authorization for convalescent plasma in August. Another cautionary tale is hydroxychloroquine, the drug President Trump began touting in March. The FDA authorized it on the basis of observational studies and later had to reverse itself when RCTs showed it to be ineffective and unsafe.
Conducting nonrandomized studies not only generates lower-quality evidence but can also divert patients, funding, and researchers’ time that might otherwise have been directed to RCTs. Every effort should be made to implement RCTs during disease outbreaks. Getting them going quickly requires planning beforehand. When the next strain of coronavirus appears, for example, which therapeutic approaches should we test and what should the RCT design be?
SHARING RESULTS QUICKLY
Is it advisable, given the urgency of learning during pandemics, to publicize study findings before they undergo peer review? During a crisis, sharing results quickly can save lives or motivate other scientists to pursue additional work or abandon dead ends, but it can also cause rapid dissemination of low-quality studies with potentially flawed conclusions.
Traditionally, study reports aren’t made public until they have been submitted to a scholarly journal, which asks experts to critique them. Journal editors then either reject the paper or require the authors to respond to questions and address reviewers’ criticisms. There may be several rounds of back and forth until the editors are satisfied that the report is ready for public consumption. This process can take months.
Even before COVID-19, science was shifting toward earlier sharing of reports. In 2019, Cold Spring Harbor Laboratory, Yale University, and BMJ, a scientific journal company, created two online platforms, medRxiv and bioRxiv, where researchers in the health and biological sciences could post “preprints” of their papers. Because journals often require authors to keep papers confidential until they’re published—a strong disincentive to share results early—the platform founders got leading journals to allow pre-publication dissemination. MedRxiv and bioRxiv now host nearly 9,000 papers related to SARS CoV-2. These early releases have played an important role in informing pandemic responses.
Scientists understand the limitations of non-peer-reviewed reports—however, others may not. Thanks to social media, preprints are being circulated quickly, widely, and with little reflection on their merits. Yet a study of COVID-19 preprints later published in journals found that more than a quarter underwent changes to the abstracts that affected the study’s conclusions.
Before physicians start changing their clinical care based on preprints, these papers should undergo at least some review by experts. The open-access journal RR:C19, launched in July, is a promising vehicle. An initiative of the MIT Press and the University of California Berkeley, it produces expert reviews of important preprints on COVID-19 in a few days. But it can only reach a fraction of the many preprints being published. Researchers have called for more scientists to volunteer for and mobilize rapid review services.
Even in a global pandemic, researchers can face a shortage of patients at research hospitals who are willing and eligible to enroll in clinical trials. When research teams are jockeying to recruit sick patients, who should get priority?
The explosion of human studies during epidemics raises the prospect that lower-quality studies could crowd out higher-quality research, or that all trials will encounter under enrollment. Unfortunately, our system of human research oversight is not designed to prioritize among studies. We need a fair, transparent process in which a multi-disciplinary committee of experts that includes representation from disease-affected communities decides whether a given trial is worth doing at a given site, what resources it will consume, what the cost to other research studies or clinical care will be, whether there’s duplication with other studies, and whether it contributes to diversity in the overall portfolio of research being pursued. Such committees should be organized at universities, the federal government and the World Health Organization as part of pandemic preparedness efforts.
Among the emerging lessons of COVID-19 is that getting the most out of a scientific surge requires planning. Creating structures to ensure wise allocation decisions and reasonable quality control during emergencies will pay dividends.
Michelle Mello, JD, PhD, is a professor of medicine and law and David Magnus, PhD, is a professor of medicine and biomedical ethics at Stanford University.