This article is a collaboration between MedPage Today and:
A new analysis of the recent TRUE-AHF trial offers strong evidence that large numbers of patients from Eastern Europe were ineligible for the trial and should not have been enrolled. The analysis doesn't change the main finding of the trial, but it does add heat to the broiling controversy over the integrity of large international trials.
The new report comes only a few days after the investigators of another heart failure trial, TOPCAT, reported that their trial had been compromised because many patients randomized in Russia and Georgia did not receive the study drug.
In response to these recent reports clinical trial investigators are calling for major overhauls and oversight to the system for performing clinical trials.
On Saturday, at the Heart Failure 2017 meeting in Paris, Milton Packer, MD, of Baylor University Medical Center in Dallas, presented new findings from the TRUE-AHF trial. The main results of TRUE-AHF were published several weeks ago. The key finding, which is not affected by the new analysis, was that short term treatment with the novel vasodilator ularitide did not improve long-term outcomes in patients with acute heart failure.
But since ularitide did produce the expected short-term effects -- systemic vasodilation, reduction in cardiac wall stress, and intravascular decongestion -- the investigators were surprised to learn that there was no significant difference in the 48 hour clinical endpoint. At the Paris meeting Packer explained that the likely explanation is that a significant number of patients in the trial should not have been enrolled, thus making it nearly impossible to demonstrate benefit.
Packer said that before the database was locked and the blind was broken, the eligibility criteria for all patients had been reviewed. 17% of patients (n =358) did not meet the entry requirements. According to Packer, nearly all of these patients "had violated one or more stability or safety criteria that had been specifically created in order to discern the efficacy or minimize the risks of the study drug."
For the short term clinical endpoint there were important differences between patients who were eligible and ineligible. In the 1,799 patients who were eligible for the trial ularitide was superior to placebo, but it was inferior to placebo in the 358 patients who were not eligible for the trial.
Problems In Eastern Europe
The explanation for this recruitment "error" is geography: In TRUE-AHF there was a dramatic imbalance in the geographic distribution of sites where ineligible patients were enrolled.
Only 2% of North American sites, 11% of Latin American sites, and 16% of western European sites had three or more ineligible patients. By contrast, 63% of sites in the Czech Republic, Estonia, Poland, and Serbia had three or more ineligible patients. Packer reported that 18 of the 19 top recruiting sites in these regions were "sites of concern."
Packer made clear that he did not think the disparity had any effect on the long term clinical outcome endpoint, since there was no hint of a benefit in any region. This means that the new finding does not have any significant clinical implications, since ularitide is not approved for use.
By contrast, the recent TOPCAT finding raised the spectre that because of the trial problems a drug with likely benefits is not being offered to millions of patients with heart failure with preserved ejection fraction for whom there are no currently available therapies with proven long-term benefits. But the new TRUE-AHF finding does contribute to the questions being raised about the performance of some sites in the geographic area formerly within the sphere of influence of the Soviet Union.
Sanjay Kaul, MD, of Cedars Sinai Hospital in Los Angeles, said that the new findings about TRUE-AHF are "concerning," but do not represent a "smoking gun" for misconduct. By contrast, the evidence for misconduct in TOPCAT was stronger, since the latest findings in that trial show that many patients in that trial never even received the study drug.
The "bottom line," said Kaul, is that "this finding following immediately on the footsteps of TOPCAT revelations raises concerns about the sloppy trial conduct (or misconduct) in some regions of the world. While globalization of clinical trial research is a positive step, proper vigilance is key in ensuring findings from such trials can be relied upon for accurate regulatory and clinical decisions."
A Recipe For Disaster
Packer spent time speculating about what might have gone wrong with the trial. He spoke in general terms about the complicated relations between the sponsors of trials, who of course pay for the trials; the executive committee, which designs and analyzes the trial; and the group -- often these days a for-profit CRO (clinical research organization) -- that actually performs the trial. (TRUE-AHF was performed by a CRO.)
Packer made the seemingly obvious point that sponsors want to have trials performed for the lowest possible cost, while CRO want to make the greatest profit. He posited the case of a "hypothetical trial that should cost 150 million Euros to execute properly." The CRO promises it can do the trial for 110 million Euros, but plans to complete the trial for 80 million Euros, therefore making a profit of 30 million Euros.
Packer said the CRO can achieve this with:
- Cutbacks on site selection and education;
- Cutbacks on monitoring, auditing and corrective action; and
- Mutually reinforcing pressures to meet recruitment targets.
"If the budget for a trial is too low and recruitment is slow, there is one easy way to fulfill goals: encouraging recruitment of ineligible patients and loosening of audit and corrective procedures," said Packer.
Packer offered a few recommendations to prevent these problems in the future. He said the executive committees should be empowered to oversee operational quality and regional event rates. In addition, source documentation for all eligibility criteria and endpoints should be maintained for all patients. Finally, he recommended: "Be wary of over-performing sites, which might be financially motivated to loosen eligibility criteria and believe that deviations are unlikely to be penalized by operations because they are meeting recruitment goals."
Bob Harrington, MD, of Stanford University, said that the picture painted by Packer is one "that all of us involved in the global RCT world recognize."
"My take is that a lot of the problems Packer describes in his presentation are with the commercial CROs. Now, my view is an admittedly biased one having been director of the DCRI (Duke Clinical Research Institute) and competed with those groups for projects and funding. The commercial CROs did a good job at the beginning of their history in elevating many aspects of clinical research. In many ways they professionalized many of the procedures and processes for doing research ... But as they became massive corporate conglomerates, they increasingly focused on squeezing monies out of the research process, including by pushing international site participation. The commercial CROs don't actually do research, they sell research services and as such they don't actually have a lot invested in quality research. In fact, one could argue that they haven't cared so much about quality because they make a lot of money selling and providing things like monitoring services. There are perverse incentives here. Ask yourself this, would there have been a private equity purchase of PPD (one of the massive commercial CROs), if there were not big profits to be squeezed out of all these purchased research services? Is this how we as a society want to answer questions critical to human health?"
Harrington said that "many of us are working hard to see if we can improve the clinical research process using new technologies and tools that will obviate (or at least massively reduce) the dependence on corporate groups selling research services. Let's get back to actually doing things that matter for patients, clinicians and the healthcare system."
Packer said that he is afraid that the wrong message will emerge from the TOPCAT and TRUE-AHF stories. He does not want to demonize the Eastern European sites. "The problem is not Eastern Europe; the problem is a lack of investigator oversight. The procedures currently in place to maintain quality control at investigative sites are not adequate. In many cases, contract research organizations are not doing the job they need to do to make sure that the right patients are being enrolled in large-scale clinical trials, and they resist guidance and pressure from the trial leadership to fulfill their responsibility. We need to create a new model where the principles and practices of trial oversight are directly responsive and accountable to the trial leadership. That is not the model that exists now for many trials. If we do not change this and if we can continue to place the blame on the wrong parties, the problem will continue. The message is really clear. This is a problem we need to solve right now. Not tomorrow, but now."
Related: