In a gigantic feat of scientific ambition, researchers have designed a staggering 1,200 clinical trials aimed at testing treatment and prevention strategies against Covid-19 since the start of January. But a new STAT analysis shows the effort has been marked by disorder and disorganization, with huge financial resources wasted.
The analysis, conducted in partnership with Applied XL, a Newlab Venture Studio company, found that one in every six trials was designed to study the malaria drugs hydroxychloroquine or chloroquine, which have been shown to have no benefit in hospitalized patients.
“If the goal was to optimize the likelihood of figuring out the best treatment options, the system is off course,” said Robert Califf, the head of clinical policy and strategy at Verily Life Sciences and Google Health and a former commissioner of the Food and Drug Administration. The findings show, he said, that too often studies are too small to answer questions, lack real control groups, and put too much emphasis on a few potential treatments, as occurred with hydroxychloroquine.
Indeed, the analysis found many of the studies are so small — 39% are enrolling or plan to enroll fewer than 100 patients — that they are unlikely to yield clear results. About 38% of the studies have not actually begun enrolling patients.
“It’s a huge amount of wasted effort and wasted energy when actually a bit of coordination and collaboration could go a long way and answer a few questions,” said Martin Landray, a professor of medicine at Oxford University and one of the lead researchers on the RECOVERY study, a large trial of multiple treatments being run by the U.K. government.
Not all effort has been for naught. Three of the most important conclusions about Covid-19 treatments so far have come from the RECOVERY trial. It has shown that dexamethasone, an inexpensive steroid, reduced the death rate of Covid-19 patients on ventilators by a third. It has also demonstrated that neither hydroxychloroquine nor a pair of HIV drugs, lopinavir and ritonavir — which had shown some early promise in laboratory models of the disease — benefit hospitalized patients with Covid-19.
Still, experts say the analysis shows that huge amounts of energy have been expended on haphazard efforts, often without a clear strategy to improve the odds that results would actually inform the care of patients. Faced with intense pressure to develop drugs and vaccines at previously unimaginable speed to push back a global pandemic, researchers may have actually slowed down the rate of progress.
For instance, 237,000 patient volunteers were to be enrolled in studies of hydroxychloroquine or chloroquine. That’s 35% of the 685,000 patient volunteers whom researchers hoped would be enrolled in any study. Since patients willing to enter studies are one of the scarcest resources in medicine, this means that other potential treatments, such as ivermectin or favipiravir, were not studied.
That’s “excessive,” said Susan Desmond-Hellmann, former CEO of the Bill and Melinda Gates Foundation. She noted that vaccines, in contrast, are being developed in a more methodical way — and wished that research on new drugs had been more organized, rather than simply trying whatever was available.
The data for the new analysis come from clinicaltrials.gov, the U.S. government database.
AppliedXL analyzed the data to identify trials that explicitly studied Covid-19 therapies and prevention strategies. The analysis focused only on interventional clinical trials, which study potential treatments or preventatives for disease. More than 880 observational studies have also been started, according to the analysis. For trials that test multiple treatments, the entire study enrollment was counted for each drug. (For more details on the methodology used in the analysis, see here.)
The analysis reflects the data as of June 24. In the days before this article was published, approximately 20 Covid-19 studies were added to the database each day.
The clinicaltrials.gov database is known to contain routine errors, oversights, and omissions, and researchers in both industry and academia often fail to update trial listings. As a result, the analysis itself involved some degree of subjectivity. But the general conclusions — both about the scale of the research and its limitations — seem indisputable.
Nahid Bhadelia, medical director of the Special Pathogens Unit at the Boston University School of Medicine, called the data “a cry for greater global collaboration during pandemics.”
America’s research infrastructure mobilized quickly when the pandemic began. In January, 10 studies were to be started, followed by 43 in February and 99 in March. In April, almost 400 studies for dozens of different treatments and preventatives were to begin, according to the database.
The sheer speed with which the studies were started was remarkable. And experts said the start of some small studies, particularly of new, experimental drugs that were previously being tested in other diseases, makes sense as a way of figuring out what might work. But such “phase 1” studies represented only 12% of the total in the analysis.
Experts added that, because the prognosis for patients with Covid-19 varies so dramatically — some patients have no symptoms, while others die on ventilators — only large studies that randomly assign patients to a treatment or placebo can deliver real insight into whether or not medicines are actually helping patients. Otherwise, researchers are fooled into thinking that differences between groups of patients with varying degrees of illness are caused by the medicines they are testing.
The RECOVERY study took a unique approach. In order to run such a large study, the researchers stripped down the amount of data collected on each patient — focusing mainly on whether patients lived or died — so that frontline researchers would be able to collect the data. More important, they got buy-in from the U.K.’s National Health Service that such a study was a priority.
Repeating that model, experts agree, would teach doctors more about how to treat Covid-19, and do so much faster.
“If more people took the RECOVERY model, or something like it, did that for the drugs they were interested in, in the patients they’re interested in, in their part of the world … we make progress an awful lot quicker,” said Landray.
Clinical trials can routinely cost $10 million or more, with some studies costing hundreds of millions of dollars. But Landray says RECOVERY was funded by a grant of about $2.5 million to the center coordinating the study, although costs at hospitals could push the total higher.
In fact, of 1,200 studies, almost all the certain knowledge — and the proof that two treatments are effective — has come from two: RECOVERY and a study conducted by the U.S. National Institutes of Health that showed Gilead’s intravenous drug remdesivir speeds the time it takes for hospitalized patients to recover from Covid-19.
Data collected by AppliedXL and STAT show that researchers had little interest in studying dexamethasone, the only medicine shown to save the lives of Covid-19 patients. There have been seven studies of the drug in all, enrolling 13,600 patients, 12,000 of whom were in RECOVERY. Two other steroid drugs, prednisone and methylprednisolone, are being studied in another 2,500 patients.
The experience with hydroxychloroquine illustrates how the research enterprise became so lopsided and unfocused. In February, hydroxychloroquine was one of several medicines that showed promise in cell cultures as a potential antiviral treatment for the virus. Although none of these medicines was designed specifically to combat SARS-CoV-2, the coronavirus that causes Covid-19, there was real reason to hope one might work as an antiviral treatment to either help infected patients or prevent infection.
Early results from researchers in France ignited interest in the drug in March, even though they were not drawn from a randomized trial. On March 19, President Trump, at a press conference, said that hydroxychloroquine had “shown very encouraging — very, very encouraging early results,” and promised “we’re going to be able to make that drug available almost immediately.” The watchdog group Media Matters said that between March 23 and April 6, guests and hosts on Fox News mentioned the medicine nearly 300 times.
Huge donations of hydroxychloroquine and chloroquine were made by generic pharmaceutical manufacturers to a national stockpile of the drugs, leading the Food and Drug Administration to grant emergency use for its use in hospitals on March 28.
By early April, 58% of patients hospitalized with Covid-19 were receiving the drug, triple the level in February, according to CarePort Health, which collects drug utilization data from electronic medical records.
Instead of conducting studies that were large enough and well-designed enough to find out if hydroxychloroquine or chloroquine could prevent or treat Covid-19, many doses went to studies without control groups, essentially tracking the use of the medicine in hospitals.
On Saturday, the World Health Organization said its own large study had found no benefit for either hydroxychloroquine or lopinavir-ritonavir. It’s still possible that one of the ongoing studies of hydroxychloroquine will show a benefit, perhaps earlier in the disease.
“The lack of leadership around a clinical trial agenda in the US is one of the failures of the US’s pandemic response,” said Walid Gellad, director of the Center for Pharmaceutical Policy and Prescribing at the University of Pittsburgh. “If we had taken the U.K.’s strategy of a set of large pragmatic trials, prioritizing recruitment in those trials, we could have all the answers now that we’re waiting for.”
U.S. researchers rushed quickly into the battle against Covid-19. What was missing, it seems, was a general who could direct them in the fight.
