Overview

The enterprise of drug development is a crucial lifeline for patients and their families. Those who need new and better treatment options depend on researchers to deliver safe and effective therapies as quickly as possible, meaning experimental drugs must first be tested on human volunteers before they can be approved for widespread use.

Since the mid-twentieth century, the randomized controlled trial (RCT) has been considered the gold-standard in research design because of its ability to overcome bias and yield high-quality evidence. But it comes at a steep cost: The average new drug requires six to eight years of human testing and $100 million to fund the clinical trial phase alone. Moreover, conducting an RCT is not always feasible or moral, such as during a pandemic or in the case of a very rare disease.

In such cases, alternative trial designs may produce faster and cheaper results, but in doing so, they must not compromise appropriate levels of standards of safety and efficacy, say regulators, patients, and insurers. While more rapid development is critical to save lives, difficult questions remain about how to tread this delicate balance.

On June 21 – 22, 2017, the Academy convened a colloquium at which academic and pharmaceutical researchers, federal regulators, bioethicists, executives, patient advocates, and lawyers met to discuss the relevance of the randomized controlled trial as the default model for human subject research. Talks focused on the history of the RCT, the ethics and use of alternative trial designs, the risks of foregoing traditional tools, the role of patient advocacy, lessons learned from a recent case study, and the importance of innovation in reforming a flawed system.

With the success of emerging interventions like genomic therapy and immunotherapy, a cultural conversation has opened up around issues such as determining how clinical trials should be designed in this new era, who may participate in research, and when promising therapies should reach the market. Formulating answers to these urgent questions could benefit millions of patients and reshape the future of medicine.

Speakers

Sponsors

This symposium was made possible with support from

Presented by

Meeting Reports

History and Contribution of Randomized Controlled Trials to Public Health

Speakers

Highlights

• American laws regarding drug testing transformed in the 1960s in response to crisis.
• New epidemics and targeted genomic therapy are prompting re-evaluation of the RCT.
• Industry sponsors have a responsibility to uphold data quality.
• A tension may exist between statistical endpoints and patient experiences.

History lessons

The opening panel set the stage for the role RCTs have played in the history of medical research. Susan Lederer, a professor of medical history and bioethics at the University of Wisconsin, described how clinical trials first came to be. In the 1760s, James Lind was a ship surgeon in the British navy faced with a rash of scurvy cases.

In a bid to stop the outbreak, he divided twelve sailors into groups of two, rotating each through different sets of treatments. The groups tried sea water, sulfuric acid, vinegar, cider, a tamarind paste, and oranges and lemons. When that last treatment proved effective, Lind realized he had hit upon a cure.

But officially randomizing treatment into a control arm and a trial arm didn’t gain traction until the mid-twentieth century, when World War II prompted a massive influx of federal dollars for research, and the pharmaceutical industry began to transform American medicine. In the early 1960s, after many pregnant women took the drug thalidomide, which caused fetal deaths and birth defects, Congress established laws calling for “adequate and well-controlled” studies that demonstrated efficacy as well as safety before drugs could be approved.

By the 1970s and 80s, the RCT had become the gold standard, said Arthur Caplan, a bioethicist at the NYU School of Medicine. But just two decades later, criticism emerged during the HIV epidemic, when many patients pushed back on ethical grounds against being randomized, contending that scientific advancement should not come at the cost of their own lives. While some patient groups praised the RCT model, some observers, like prominent physician Marcia Angell, called into question researchers’ “slavish adherence” to the RCT at the expense of compassion for individual sufferers.

In the current era of epidemics like Ebola and Zika, and the increasing prevalence of targeted genomic therapies, the relevance of the standard RCT has been called into question with renewed urgency. Some situations, Caplan said, don’t permit the time or expense of a standard RCT. That’s a point which has raised substantial debate, but some argue that patients may be too sick to participate, the need for treatment may be immediate, the number of sufferers too small, or the ability to maintain oversight too unrealistic.

Jane Perlmutter, a patient advocate, offered additional concerns about RCTs, including limitations on generalizability if trial subjects don’t comply with the protocol, and if eligibility requirements narrow the scope of the testing population.

“This gold standard is not so terrific,” she declared. “We need to innovate.”

Patients also don’t like to be guinea pigs, she noted, suggesting one “baby step,” towards innovation could be allowing patients to choose to join the experiment or control group, rather than be blinded and randomized. Still, randomizing some patients in is important to preventing bias and leading to generalizable findings. In such a design, researchers must carefully assess the data to ensure that the randomized subjects and the self-chosen subjects show no misleading discrepancies. As long as their results align, this type of study can be both efficient and effective.

At the same time, Howard Fingert, senior medical director at Takeda Pharmaceuticals, said that the management of big data is a major responsibility and opportunity for industry sponsors no matter the trial design. Mechanisms for data sharing can improve understanding in trials that are single-armed, propensity-matched (meaning those driven by a statistical score that estimates a treatment’s effectiveness), or underpowered (meaning those statistically unlikely to distinguish a treatment effect from pure luck due to a low sample size).

And upholding data that is reflective of reality is crucial, he said. For example, if a primary endpoint in a study fails, investigators may look for a positive secondary endpoint that wasn’t originally in the protocol—a practice he called “ubiquitous but not legitimate.”

Caplan asked whether fear of adverse outcomes is hindering innovation, citing the 1999 death of an 18-year-old in a clinical trial for gene therapy, which proved a major setback for the entire burgeoning field. Susan Ellenberg, a professor of biostatistics at the University of Pennsylvania, noted that risk aversion is even more common now, as social media reinforces people’s negative beliefs about the prevalence of dire medical events, like vaccine toxicity. But 37 states have passed right-to-try laws, Perlmutter pointed out, which permit terminally ill patients to volunteer for experimental therapies.

Finally, the panelists discussed how the very notion of efficacy has evolved. Decades ago, patients were asked if they were feeling better after an intervention, but now the focus is on objective endpoints—sometimes to a fault, according to Perlmutter. She questioned whether an outcome such as a shrinking cancer tumor, for example, is truly meaningful if the patient’s quality of life remains unaffected. Caplan reminded the group that in dialysis programs for end-stage renal disease in the 1970s, one measure of success was whether patients could return to work. But Ellenberg added that patient improvement and high-quality data are not mutually exclusive.

“Looking for objectivity doesn’t exclude those endpoints,” she said.

Introductory Remarks and Panel Discussion

Further Readings

Beyond RCTs—Assessing the Need for Alternatives

Speakers

Highlights

• Choosing between a standard RCT and a non-RCT alternative can be a false dichotomy.

• FDA states that all patients deserve the same evidentiary and regulatory standard.

• Tension exists between targeting therapies to the right patient population and understanding drug safety and efficacy.

Innovation without compromise

Moderator Charles Weijer, a bioethicist at Western University, kicked off the discussion by asking panelists to offer up important lessons about clinical trials from the past. Barry Gertz, a partner at Clarus, pointed to randomization as the key to reducing bias, and a reason why the RCT ought to remain the default design for testing new agents. Despite its heavy costs, he argued, the overall societal burden would be even higher without it, citing the example of a new device tested for severe hypertension. It produced what seemed at first to be miraculous results—until a subsequent RCT proved it no better than a sham procedure.

“If you don’t test with adequate rigor,” he warned, “society will pay a very substantial price if it’s not as effective as it’s billed to be.”

Luciana Borio, acting chief scientist at the FDA, agreed that classic RCTs still deserve a primary place in the medical ecosystem.

“Nobody has said, ‘I regret doing an RCT,’” she said. “History has played out the other way around; ‘we didn’t know and had to live with the consequences.’”

Even in cases of public health emergencies like Ebola, when some scientists deem RCTs impractical, she maintained that such situations especially demand informative studies.

“We have to be better prepared for the next epidemic,” she added. “We can’t say it’s too hard to do it.”

Weijer commented on the importance of avoiding a false dichotomy between RCTs and alternatives that are still capable of incorporating randomization into their designs. For example, Borio mentioned the ring study carried out during the Ebola crisis, in which groups of people known to be in contact with a patient—those forming a so-called “ring” around the infected person—were randomized to receive a new vaccine either immediately or after a three-week delay.

Andrea Troxel, a professor of population health and biostatistics at NYU, agreed that randomization in some capacity is necessary for the generation of high-quality knowledge, even if a standard RCT “is not always the answer.”

Weijer then raised the complex issue of whether rare diseases necessitate a lower standard of evidence for drug approval, given the lack of patients available for clinical trials. While Troxel said that “sufficient evidence” would not be possible to attain in a standard RCT, others disagreed. Gertz pointed to the approval of a drug for spinal muscle atrophy, the most common genetic cause of early demise in infants. An RCT was carried out on just 81 patients using a sham placebo administered to the spinal fluid, with results showing that the drug yielded profound increases in motor activity.

“It benefited a very small number, but even in that rare disease, the RCT provided some real evidence of benefit and is now available,” he said.

Regarding common diseases, the panelists disagreed on whether a social imperative to accelerate development justifies a lower evidentiary standard. Borio cautioned against the temptation to take short cuts, and said that all patients deserve the same high regulatory standard, while Troxel similarly warned of the “unintended consequences” of rushing new therapies out to desperate patients.

But at that moment, patient advocate Jane Perlmutter spoke up from the audience, declaring that patients with terminal illnesses want to take risks. “I don’t think we need to lower the bar, but we need to have innovative approaches to deal with deadly diseases,” she said. Gertz suggested that one potential solution would be for the FDA to grant a drug provisional approval using an intermediate or surrogate endpoint, with later testing to confirm the findings.

But if those findings failed to hold up, withdrawing such a drug from the market would be difficult, due to a backlash from patients still demanding access to the drug. He said this scenario has happened once in oncology and “it wasn’t pretty.”

In the unique case of biologically targeted therapy, Weijer posed the intriguing question as to whether the science has evolved to such point that an RCT is not needed to evaluate a drug’s effectiveness. Gertz acknowledged that studies with driving mutations in oncology are typically single arm, but that a control group is implicit in the standard of care response rates. He also maintained that a randomized trial would be needed eventually to determine safety as well as efficacy.

All the panelists agreed that helping the diverse range of patients who exist in the real world is crucial, and requires testing beyond narrow subsets who don’t reflect the larger population. More recently, pragmatic trials—which more realistically mimic day-to-day practice settings—are gaining traction, said Troxel.

Such trials aim to clarify how effective a recommended therapy might be, as opposed to explanatory trials, which aim to elucidate mechanisms of action in a new agent. Yet, “those two goals are not necessarily in conflict,” she said. A trial might be explanatory at first, with small numbers and a strict inclusion criteria, then broadened to test the therapy on a wider group.

Borio lamented that researchers don’t learn from most patient encounters because of a lack of access to studies. Today, the practice of medicine often takes place separately from the world of research, so when sick patients visit the doctor, their cases are not analyzed to improve the effectiveness of treatments for others. And those who do participate in studies often fail to represent important demographic subgroups.

Currently, the typical participant is a young, white male who lives near a metropolitan area. Borio would like studies to systematically include patients who are not usually included, like those in rural areas, minorities, babies, and pregnant women. Her dream is for every person in the medical system to be able to enroll in a clinical trial.

“There’s no national trial infrastructure in this country, like highways,” she said. “But we need it so we can make the most use of all the knowledge.”

Doing so, however, she acknowledged would require a major shift in how doctors are educated about clinical trials and how patients view the riskiness of participating in research. Many patients decline to participate because they view research as inherently riskier than regular medicine, which is not necessarily true.

Panel Discussion

Further Readings

Finding the Right Balance in Learning about Therapies

Speakers

Highlights

• There are clear benefits to combining research with clinical practice.

• RCTs and alternative trials could be combined and run continuously.

• The current clinical trial system is deeply flawed and too expensive.

• Data sharing among health care systems will speed evaluation and development.

Asking the right questions

The future of human experimentation is at a crossroads. Sick and dying patients need treatment options as quickly as possible, but rushing out new therapies will not necessarily benefit them.

The real issue, said keynote speaker Robert Califf, is how to accelerate drug development but also “get it right.” The current clinical trial enterprise has “gone awry,” he said, calling it unnecessarily expensive.

Califf, a cardiologist and former commissioner of the FDA from 2016 to 2017, charged the system fails patients by not asking all the important questions. “It’s not that clinical trials are too hard, it’s that there are questions not even being asked because it’s so costly,” he said.

A schism has developed in medicine, between those who think human experimentation should be conducted within the context of daily practice and those who feel it should remain separate from it. In Califf’s view, human experiments benefit when combined with insights of clinicians, but the layers of oversight and the risk of punitive action dissuades doctors from participating in research. He criticized what he called common myths about RCTs: that they must exclude patients who represent the likeliest use of therapies, and that clinical trials are risky compared with routine care.

“There’s no reason you can’t enroll real-world clinical patients in a trial,” he said.

He also dismissed the notion that doctors regularly review evidence and make the best decisions for their patients—simply because statistically valid evidence often doesn’t exist in medicine. In fact, he said, many practice recommendations are not based on high-quality evidence, such as the CDC’s recent guidelines for prescribing opioids. Unlike the rigorous approach codified in research studies, much of medical practice is rooted in observational and historical data, leaving doctors with a limited set of tools.

His solution is to run RCTs in combination with alternatives, such as pragmatic trials, to reduce the cost and enable better generalizability from the start. In the early phases of therapeutic development, he suggested randomizing from the first patient — “the quickest way to get treatment to patients even with rare diseases.”

Then, in later phases, every interaction with patients would be logged in a digital database. He urged a national paradigm shift toward the sharing of such data across health care networks. Such an effort would be in keeping with the drive to create incentives for health systems to work together that was written into the 21^st Century Cures Act, which was signed into law in December 2016, and the user fee reauthorization bill currently working its way through Congress.

He envisions moving away from inefficient one-off studies, done in a parallel track to clinical practice with passive surveillance, to active surveillance, for instance by collecting information on millions of patients in a central database embedded in the health care system, with broad data-sharing among providers. He discussed his involvement in PCORnet, the National Patient-Centered Clinical Research Network, which collects data across hospitals, doctor’s offices, and community clinics in an attempt to help guide healthcare decisions.

He also urged patients to push academic health systems to stop hoarding data, positing that if the medical world shared the business world’s mentality of persistent data collection, progress would accelerate.

“When you do a Google search, you’re participating in up to ten randomized trials,” he noted.

The bottom line, he concluded, is not to abandon RCTs, but to maintain continuous and constant observation as health care is delivered. Just as Google analyzes its data nonstop to improve user experience and anticipate search queries, medicine ought to catalogue and interpret its abundance of real-world data to bring to light the best treatment options for patients.

Keynote Presentation and Audience Q&A

Further Readings

Weighing the Risks of Randomized Controlled Trials and Alternatives

Speakers

Highlights

• Controversy surrounds the use of RCTs during public health emergencies.
• Platform trials can reduce costs and increase efficiency.
• Vulnerable populations need adequate access to clinical trials.
• Increasing research participation is key to obtaining comprehensive data.

Maintaining equipoise

Genuine uncertainty about the comparative effectiveness of different interventions is the ethical foundation for randomized clinical testing, a concept known as equipoise. Moderator Steve Usdin, Washington editor of BioCentury, opened the discussion by asking panelists to weigh in on the challenges of striking equipoise during therapeutic development. If a given intervention is known to work, the researcher cannot in good conscience withhold it from test subjects, said Harvard bioethicist Holly Fernandez Lynch.

Robert Walker, acting chief medical officer of the Biomedical Advanced Research and Development Authority (BARDA) within the U.S. Department of Health and Human Services, discussed the challenges of maintaining equipoise during the Ebola crisis.

“There was a sense that you can’t conduct a clinical trial in the midst of an emergency response, but we saw that it was in fact feasible,” he said, citing three randomized vaccines trials carried out in Liberia, New Guinea, and Sierra Leone.

As to the private groups that granted some patients emergency access to treatment during the public health emergency—without a trial to gauge efficacy—Walker said, “It’s not even information…We really didn’t learn.”

Usdin asked the panelists to describe when to use RCTs versus alternatives. Amrit Ray, chief medical officer at Johnson & Johnson, posed a solution that would retain the benefits of randomization but reduce the costs and facilitate data sharing: integrated platform trials. In the current system, five companies might test five different drugs for the same disease in isolation. Instead, Ray proposed, what if those companies collaborated on one joint trial with a common control arm? This would lessen the burden of duplicate trials and patient recruitment, and allow for faster data collection to evaluate drugs, as with the innovative I SPY-2 trials in breast cancer.

Then Matthew Rotelli, a director at Eli Lilly, raised the challenge of how to broaden clinical trials to include vulnerable populations like children, “because if you don’t study them,” he said, “you have no way to guide their treatment.” Walker responded that BARDA has a legislative mandate to study all populations, and that special additional oversight for kids would only make the process more onerous. He stated that existing measures, including institutional review board review, are already responsible for ensuring proper informed consent.

Regarding novel trial designs, Usdin worried that even if the data persuades regulators to approve a drug, insurers still might not pay without the legitimacy conferred by a standard RCT. Ray responded that the comparison design of platform trials could potentially mitigate that risk.

Another new paradigm could emerge to meet serious unmet needs—allowing patients to risk taking a promising experimental drug faster in exchange for the sponsor collecting comprehensive data post-market. While Fernandez called this an “ideal world,” she was somberly realistic about its prospects. New laws would have to be passed, and sponsors would need to be held accountable.

Rotelli envisioned a future in which clinical trials never end. As new drugs come out, they are added to ongoing randomized platform trials for further study against known drugs. Less effective ones eventually get dropped, while electronic medical records facilitate the data collection. But obtaining and sharing that data would require a “dramatic shakeup of systems,” Fernandez said.

Right now, research participation requires robust informed consent, with autonomy prized as the highest value. It operates in a separate layer from clinical care, protected by institutional review boards that regularly review protocols to protect participants from exploitation.

An opposite model, in which data collection is routine, would require most patients to participate by default, rather than choosing to opt in.

“I don’t know that I would go to that end of the spectrum,” she said. “It’s so different from how we’ve done research, given concerns about historical abuses.”

Expecting compulsory participation makes many observers in the medical community uneasy. The twentieth century, after all, is filled with brazen examples of vulnerable subjects who were harmed or killed for the sake of science, including concentration camp victims and the cohort of African-American men infected with syphilis who, unbeknownst to them, were denied penicillin by Tuskegee researchers. Such infamous cases led to establishment of a set of morals and rules for human research participation via the 1947 Nuremberg Code and the 1979 Belmont Report, establishing the ethical pillars of autonomy and informed consent.

Unless such historical abuses fade in the collective consciousness, any future reforms that dial back the protection of individuals are unlikely to be popular.

Panel Discussion

Further Reading

Ethics and Patient Advocacy in Clinical Trial Design

Speakers

Highlights

• A cultural clash exists between patients and researchers.
• Many specialists agree that clinical design could be improved by involving patient experts early on.
• Early access programs allow patients to try experimental therapies outside of clinical trials.
• Expanding inclusion criteria would allow more people to participate in research.

Balancing acts

When it comes to the policies and guidelines that govern drug development, patients and researchers often find themselves at odds, clashing over aspects such as trial design, compliance protocols, and early access. Differing motivations lie at the heart of the conflict, suggested moderator Alison Bateman-House, a bioethicist at NYU. Researchers want to help push science forward, while patients want access to therapies that will help them and their loved ones.

“When you’re a patient or a parent, [those with] the option of being in a trial think, ‘What would benefit me most?’” said Rebecca Susan Dresser, a cancer survivor and bioethicist at Washington University in St. Louis. “Altruism is low on the list.”

Many of those who can’t participate in trials struggle to persuade sponsors to let them try experimental therapies under the FDA’s program of expanded access, also known as compassionate use. Companies may be reluctant to participate in compassionate use in part because giving patients a drug outside of a trial means a lost opportunity to gain valuable data on safety and efficacy.

Advocacy groups, such as Andrew McFadyen’s Isaac Foundation, fight for access despite these concerns, because offering patients with rare diseases otherwise unavailable treatments can mean the difference between life and death.

“While RCTs are good at getting approval of drugs, it’s a hindrance in getting them to our children,” he said.

Patients and researchers should work together from the start of designing a trial so their goals can coincide, the panelists all suggested. Too often, Bateman-House said, patients are silent partners, which can translate into a protocol that’s too lengthy, intensive, or inconvenient. For example, Dresser rejected an offer to join a cancer trial for her advanced illness because its timeline would have meant a delay in starting treatment.

Jane Reese-Coulbourne, a former cancer trial participant and now consultant, stressed the importance of companies bringing in expert patients for feedback. As a patient advisor, she helped Genentech, a major biotech company responsible for several dozen pioneering drugs, understand why its recruitment for one particular trial was so low: the protocol required enduring four painful bone marrow aspirations, a procedure which extracts fluid from the marrow.

But when collaborating, patient advocacy groups can sometimes find themselves toeing a tricky ethical line between accepting funding from pharmaceutical companies and maintaining their organizational independence.

“Often pharma does bring us in for advice,” McFadyen said, “but sometimes the expectation there is that we’re going to be the people out there making sure [the drug] gets reimbursed.”

Some advocacy groups, he said, will accept payment in exchange for not asking tough questions about access. J. Russell Teagarden, a pharmacist, executive and educator, acknowledged that there are advocacy groups vulnerable to coercion.

“But on the other hand, there are some groups so sophisticated that maybe the companies are vulnerable,” he said, to laughs.

Another source of conflict between companies and patients is rhetoric. Too often, companies employ hyperbolic language like “breakthrough” to attract investors, while desperate patients line up for trials that haven’t in actuality even begun.

Bateman-House said that the lack of scientific literacy in the general population, who are prone to believing the science holds all the answers, is a major source of confusion. In fact the vast majority of drugs fail in the lab long before the point of human trials. Indeed, only one in a thousand compounds graduates to clinical testing, and of those, only about 10 percent eventually cross the finish line.

Issues about trial demographics came to the fore during a vibrant Q&A. One audience member expressed concern over how to expand clinical trials to underrepresented groups, like pediatric cancer patients. McFadyen suggested that is possible if parents push for it. Another spoke of disappointment over the fact that only 10 percent of patients qualify for trials. Several panelists responded that legislation is underway in the Senate to expand inclusion criteria so more people can participate in research.

Panel Discussion

Further Readings

Modern Trends in Clinical Drug Development

Speakers

Highlights

• Clinical trials are a prohibitively expensive element of drug development.
• Trial designs, no matter how novel, will only be as good as the knowledge underlying them.
• Randomization remains an important tool but is not always necessary in novel designs.
• The system requires reform to incentivize continuous, collaborative platform trials.

Designing alternatives

The clinical trial system in the United States is broken because it isn’t “fit for purpose,” argued Janet Woodcock, director of the Center for Drug Evaluation and Research at the FDA. Because trials are so expensive and time-consuming, many questions remain unanswered after a drug is approved, leading to health care practices that too often lack high-quality evidence.

She discussed the wasteful efforts when a sponsor sets up a trial, tests an intervention, and then walks away because either the trial fails or the drug reaches the market. Instead, the future ought to bring more continuous, ongoing platform trials that can answer multiple questions at once, with data shared among health care networks.

“The goal,” she said, “is not to test a specific therapy, but to bring about continuous improvement in disease outcomes.”

In the current era, rapidly evolving science is driving novel research designs. Molecularly targeted therapies are on the rise, along with drugs that are “disease agnostic,” such as those that might target a specific biomarker appearing across tumor types, for example. With rare, life-threatening diseases that lack treatment options, she suggested it may in fact be adequate to test a targeted therapy, which is expected to show a large treatment effect, in a single-arm trial with an extended phase one cohort.

“I’m going to say something heretical, but oncology has been doing this, and it’s perfectly reasonable in my mind under these circumstances,” she said.

While a very useful tool, randomization in her mind is not an imperative. However, “it’s foolish not to use it if at all possible,” she added, such as if the disease outcomes are very variable or researchers don’t expect to get a homerun treatment effect.

But as more development programs are working on very rare and orphan diseases, the FDA is approving drugs based on limited trials that may lack randomization, as with an antidote for methotrexate toxicity devised from the data of 22 patients. Such cases involve a serious unmet medical need, a well-understood disease, highly plausible biomarkers that can be easily measured in a standardized way, and a drug that yields a large treatment effect.

She cautioned, however, that the design of a trial is only as good as the quality of the knowledge underlying it. Biomarkers may mistakenly drive a development program, for example, if they are not reproducibly measured, accurate, or predictive. Before starting trials in humans, researchers should be confident that a biomarker is “reasonably likely to predict clinical benefit,” she said, suggesting that randomization usually remains the best design in this situation.

Calling for greater efficiency overall, she urged a shift toward collaborative platform trials that integrate research and practice. But implementing changes to the status quo poses a serious challenge.

“In this translational world of continuous improvement in medicine, nobody is charged to do it and that’s the real problem,” she acknowledged. Since there isn’t a key stakeholder, she urged patients to rise up and demand reform.

Keynote Presentation and Audience Q&A

Further Readings

Lessons from the Eteplirsen Drug Trial for Duchenne Muscular Dystrophy

Speakers

Highlights

• Patients-researcher collaboration could enhance planning of clinical trials.
• Surrogate endpoints should comport with clinical gains.
• Understanding a disease’s natural history and its biomarkers is crucial to guiding high-quality research.
• Limited trials can lead to accelerated approvals, but questions may remain about real-world effectiveness and who will pay for the drug.

Case study

The accelerated FDA approval late last year of the drug Eteplirsen for Duchenne Muscular Dystrophy (a disorder that predominantly manifests in young boys and progresses rapidly) based on controversial data from a 12-patient randomized trial, set off a lively discussion. Sufferers with this rare disease progressively lose muscle function and previously had no treatment options before Eteplirsen came on the market in late 2016, gaining approval even though its effectiveness is still being debated. Moderator Meg Tirrell of CNBC opened by asking the panelists whether the case established any precedents.

Ellis Unger, director of Drug Evaluation-I in the Office of New Drugs at the FDA, offered his concern about the trial’s surrogate endpoint: the drug was approved based on a small increase in the amount of dystrophin, a key protein, found in skeletal muscle. However, whether the increase yields meaningful clinical benefits remains an open question and a flash point for controversy. Unger said he worried that the case will prompt other companies to present similarly limited data and expect to gain FDA approval.

But Pat Furlong, a patient activist and the mother of two boys who died of the disease, argued that the drug did show improvement in the gaits of those who took it, and that desperate patients should not be prevented from taking risks. “It has rocked my world,” she said, speaking of the hope it’s brought to patients who now have the option to try a drug where before none existed.

Edward Kaye, CEO of Sarepta Therapeutics, the drug’s sponsor, contended that small amounts of a biologically active component can work, and said that “the bigger precedent is patient involvement.” Throughout the study, his company worked with patients’ families to understand what quality-of-life outcomes would be most meaningful from a drug, such as the ability to go to the bathroom independently. He also collaborated with other companies to accelerate the search for biomarkers. Their collective pact to publish joint findings represents a notable shift in how research is done.

“It’s not one company against another,” he said, “it’s a number of companies and patient groups against the disease.”

In response to a question about the importance of biomarkers in raising capital for research, David Scheer, an entrepreneur in the life sciences, said that they provide crucial preclinical data.

“Without having some sort of biomarker technology that can facilitate translational medicine, we might be shooting in the blind,” he explained.

The discussion turned heated when Unger described the unusual public comment period during an FDA advisory meeting prior to the drug’s approval. While some of the boys from the trial declared their improvement, Unger said the data showed they were in fact deteriorating. He also described the comment period as “a circus,” because in his view patients and families went over the line, taking too long, and in some instances heckling the committee.

Shortly thereafter, patient advocate Andrew McFadyen approached the microphone and admonished Unger. McFadyen told him to show more respect for families with dying children and to listen to their stories for a year if necessary. “If you can’t do that, you should give your chair up to someone else,” he declared.

While Unger apologized for using the word “circus,” he defended his remarks, recounting the conduct of the session.

“When the deputy director of the Neurology Division told a very personal story of tragedy, he was heckled.” He continued, “we have to draw a line somewhere, and we thought that the time we allotted was reasonable… the catcalls, and the heckling, it was very disheartening.”

Former FDA commissioner Robert Califf stepped up to offer additional insight into the process. “Advisory committees are so named because they do not make the decisions,” Califf said. “Full time government employees are the ones who make the decisions and most people are still confused by that.”

“There was no one in the FDA who thought that the studies were well done,” he added. “I don’t think these decisions would have been so hard otherwise.”

Another audience question addressed the cost of the drug: At a price of $300,000 annually per patient, insurers are balking; they want to see real-life evidence of effectiveness before paying for it. Kaye’s company has started a registry to gather such data, but he said that new therapies in small populations entail expensive drugs because limited numbers of patients receive them.

“So, the cost of developing the drug is transferred to them,” he said. “That is the cost of innovation.”

Panel Discussion

Further Readings

A Way Forward—Shaping Clinical Trial Innovation

Speakers

Highlights

• Platform trials are a key way to merge research and practice.
• RCTs are still considered the definitive trial design for reducing bias.
• Sharing well-established trial templates for common diseases will eliminate redundant efforts.
• Sponsors should make compliance less burdensome for patients.

Refinements or reinvention?

The lively panelists in the afternoon session had no qualms about offering blunt criticism of the status quo. Echoing a theme of the conference, moderator Donald Berry, a statistician at MD Anderson Cancer Center, underscored the need to merge research with practice.

“It’s inevitable,” he said. “We have to figure out how to do it, and platform trials may be the way.”

George Demetri, a medical oncologist at the Dana-Farber Cancer Institute, summed up the situation more derisively: “The clinical trial system is broken,” he said, “and we’re not being honest with the public about what we do and don’t know.” He argued that the benefits of precision medicine and testing for mutations are oversold to patients, particularly by academic institutions.

Christopher Robertson, a law professor at the University of Arizona, raised concerns about bias skewing results, especially in cancer drugs with small effect sizes. Blinding not only patients, but also investigators and statisticians, is crucial to finding legitimate outcomes. Another way to reduce bias is to specify endpoints before the start of a trial to avoid coming up with erroneous probability values, he said.

Anne Cropp, chief scientific officer of Early Access Care, suggested that one way to improve efficiency in the development of new protocols is to make data from the NIH more widely available so that companies working on common diseases like Alzheimer’s and diabetes can use established trial templates, with well-defined endpoints, rather than starting from scratch. She also urged physicians to become more literate in drug testing, citing a Tufts study that revealed a surprising lack of knowledge among doctors regarding the ins and outs of clinical trials.

John Thompson, a professor of biostatistics and neurology at Columbia, cautioned against a premature rejection of double-blind, placebo-controlled RCTs. He worked on just such a trial involving a treatment for ALS, despite being told it was impossible because patients would decline to participate.

“Must RCTs give way?” he asked. “A flexible no, although many people will hear it as a yes, because I envisage considerable fast-moving changes.…It’s a matter of adding to and expanding rather than abandoning the existing techniques.”

A key to the ALS trial’s success, he said, was explaining to patients the importance of its design to gain their trust and cooperation. To help with recruitment, he suggested that disease advocacy groups post the details of ongoing trials on their websites, after networking with investigators, so that patients can find the information and take it to their doctors.

During the audience comment period, Ellis Unger of the FDA offered additional suggestions. He mentioned a “very powerful” trial design that he believes is not used enough: randomized withdrawal, in which responders stop taking a drug and investigators observe what happens. He also urged sponsors to make compliance easier for patients by letting them participate from home, via Skype, when possible, and to limit the number of scans and tests they must undergo.

“The FDA does not need forty thousand blood tests or four X-rays per person to approve a drug,” he said dryly.

Panel Discussion

Further Readings

Ethics Panel Wrap-up

What is the Future of Accelerated Development and the Randomized Controlled Trial Standards?

Speakers

Highlights

• RCTs remain a key method for overcoming bias and threats to validity.
• But not every drug requires a large, expensive RCT for initial approval.
• Platform trials retain randomization while operating more efficiently.
• The system requires reform to incentivize collaboration and transparency.

One tool along the continuum

The final presenters agreed that RCTs are not going away anytime soon, but there are changes in how and when they are being used.

Vinay Prasad, a hematologist-oncologist at Oregon Health and Sciences University, defended the RCT against common criticisms. If such trials don’t reflect real-world populations, he said that is due to strict inclusion criteria, not the design. And if endpoints don’t represent clinical outcomes, the RCT itself is not to blame. That would be “like blaming the Wright brothers for United Airlines,” he quipped.

Eric Rubin, vice president of oncology clinical research at Merck, argued that the title of the conference should not be, “Must RCTs give way?” but rather, “Should RCTs not get in the way?” For example, a single-arm study in a drug with a large effect size can lead to initial approval faster than a standard RCT. Finding such a drug requires high-quality basic research and reproducible preclinical studies, he added.

Jeffrey Weber, deputy director of the Perlmutter Cancer Center at NYU, acknowledged that everyone “worships the god of the randomized, phase three trial,” but he expressed concern over its exploding costs and suggested that modifications in some cases could be appropriate. He proposed using more novel endpoints, such as landmark survival at one and two years, combined with a quality-of-life questionnaire.

“In the immunotherapy era,” he said, “I think we can afford to be more flexible than to do an RCT with a thousand participants and the only endpoint is survival.”

But Prasad was quick with a rebuttal, arguing that progression-free survival is based on an arbitrary line in the sand and is not necessarily correlated to eventual survival.

Rubin agreed with Weber that RCTs are the definitive way to demonstrate benefit, but that at least in oncology, an RCT with overall survival as the endpoint is not required to approve a drug. Rubin suggested that it is possible to deliver promising therapies quickly, without sacrificing randomization, by conducting early phase explanatory trials for initial registration and then a post-market RCT to verify early results.

For example, at Merck he led the development of a melanoma drug that was approved after a single-arm study in patients with advanced illness who were out of options. Afterward, Merck followed up with a randomized trial in less sick patients to compare the new drug against existing alternatives, in which case researchers still retained equipoise. This scenario shows it’s possible to do both—accelerate approval, in this case by three years, and still conduct a confirmatory RCT.

Rubin reiterated an idea that got a lot of play across the conference—the notion that a collaborative platform trials could streamline the process of matching drugs to patients across a spectrum of disease, as was the case in I-SPY2. Such trials benefit patients by letting them get assigned to one arm no matter their cancer.

But Nancy King, a professor of social sciences and health policy at Wake Forest, pointed out that academic institutions and businesses aren’t structured to reward transparency and collaboration over competition.

“You have to be able to turn the battleship,” she remarked.

In a discussion on the exorbitant expense of running trials, Weber said that the regulatory scrutiny has proliferated over the last decade, adding to the cost burden.

“We have monitors that monitor the monitors,” he joked.

Ultimately, even as some areas of drug development move away from the RCT, it remains a fundamental tool along a continuum of designs. One reason it may never vanish is that it can act as a bulwark against the fallibility of human nature.

“Our capacity for hope makes us incredibly susceptible to inferior levels of evidence,” Prasad reflected.

In his opinion, the beauty of randomization is that with a modest effect size, there is no better way to tease apart what works.

“That’s why it’s here to stay,” he said.

Panel Discussion

Further Readings

Open Questions

• How can researchers and industry sponsors be incentivized to collaborate on platform trials and sharing data?
• How can researchers accelerate drug development without sacrificing the collection of data on efficacy and safety?
• How can society increase scientific literacy among the public to encourage participation in research, rather than view it as inherently risky?
• How can the medical profession systematically integrate research and clinical practice?
• If a drug is granted accelerated approval after a single-arm trial, how can the sponsor be held accountable for collecting post-market data on safety and efficacy?
• Will insurers pay for drugs that are approved based on limited or non-randomized trials?
• During the next humanitarian emergency, is it feasible and ethical to test interventions with a RCT?
• How can costs be better managed to complete a RCT without the prohibitive expenses?
• Can decreasing the amount of oversight streamline trials without sacrificing protection for research participants?
• Can sponsors and patients work together to design trials at the earliest stages, ensuring that compliance in a protocol is not overly burdensome?
• Should a standard RCT remain the default design?
• Is choosing between a RCT and an alternative design a false dichotomy?
• How can trials better accommodate broad patient populations to yield real-world data and integrate more patients into research?
• Who should decide how much risk is appropriate in trying an unproven therapy?
• Will big data solve quandaries of speed and the historical limits on the RCT?
• Should rare diseases with limited numbers of patients available for trials have a lower evidentiary standard for drug approval?
• What can be done to reign in exorbitant drug prices?
• Will automation and electronic medical records help motivate doctors to participate in large-scale practical trials?
• How can society implement the changes required to modernize the clinical trial enterprise?