Will Covid-19 vaccines save lives? Current trials aren’t designed to tell us | The BMJ

Safety and side effects

History shows many examples of serious adverse events from vaccines brought to market in periods of enormous pressure and expectation. There were contaminated polio vaccines in 1955, cases of Guillain-Barré syndrome in recipients of flu vaccines in 1976, and narcolepsy linked to one brand of influenza vaccine in 2009.1819

“Finding severe rare adverse events will require the study of tens of thousands of patients, but this requirement will not be met by early adoption of a product that has not completed its full trial evaluation,” Harvard drug policy researchers Jerry Avorn and Aaron Kesselheim recently wrote in JAMA.20

Covid-19 vaccine trials are currently designed to tabulate final efficacy results once 150 to 160 trial participants develop symptomatic covid-19—and most trials have specified at least one interim analysis allowing for the trials to end with even fewer data accrued.

Medscape's Eric Topol has been a vocal critic of the trials' many interim analyses. “These numbers seem totally out of line with what would be considered stopping rules,” he says. “I mean, you're talking about giving a vaccine with any of these programmes to tens of millions of people. And you're going to base that on 100 events?”8

Great uncertainty remains over how long a randomised trial of a vaccine will be allowed to proceed. If efficacy is declared, one possibility is that the thousands of volunteers who received a saline placebo would be offered the active vaccine, in effect ending the period of randomised follow-up. Such a move would have far reaching implications for our understanding of vaccines' benefits and harms, rendering uncertain our knowledge of whether the vaccines can reduce the risk of serious covid-19 disease and precluding any further ability to compare adverse events in the experimental versus the placebo arm.

“It'll be a decision we'll have to take at that time. We have not committed one way or another,” Moderna's Tal Zaks told The BMJ. “It will be a decision where FDA and NIH will also weigh in. And it will be probably a very difficult decision, because you will be weighing the benefit to the public in continuing to understand the longer term safety by keeping people on placebo and the expectation of the people who have received placebo to be crossed over now that it has been proved effective.”

Not enrolling enough elderly people or minorities

A vaccine that has been proved to reduce the risk of symptomatic disease by a certain proportion should, you might think, reduce serious outcomes such as hospital admissions and deaths in equal proportion.

Peter Marks, an FDA official with responsibility over vaccine approvals, recently stated as much about influenza vaccination, which “only prevents flu in about half the people who get it. And yet that's very important because that means that it leads to half as many deaths related to influenza each year.”24

But when vaccines are not equally effective in all populations the theory breaks down.

If frail elderly people, who are understood to die in disproportionate numbers from both influenza25 and covid-19, are not enrolled into vaccine trials in sufficient numbers to determine whether case numbers are reduced in this group, there can be little basis for assuming any benefit in terms of hospital admissions or mortality. Whatever reduction in cases is seen in the overall study population (most of which may be among healthy adults), this benefit may not apply to the frail elderly subpopulation, and few lives may be saved.

This is hard to evaluate in the current trials because there are large gaps in the types of people being enrolled in the phase III trials (table 1). Despite recruiting tens of thousands, only two trials are enrolling children less than 18 years old. All exclude immunocompromised people and pregnant or breastfeeding women, and though the trials are enrolling elderly people, few or perhaps none of the studies would seem to be designed to conclusively answer whether there is a benefit in this population, despite their obvious vulnerability to covid-19.

“Adults over 65 will be an important subgroup that we will be looking at,” Moderna's Zaks told The BMJ. “That said . . . any given study is powered for its primary endpoint—in our case covid-19 disease irrespective of age.”

Al Sommer, dean emeritus of the Johns Hopkins School of Public Health, told The BMJ, “If they have not powered for evidence of benefit in the elderly, I would find that a significant, unfortunate shortcoming.” He emphasised the need for “innovative follow-up studies that will enable us to better determine the direct level of protection immunisation has on the young and, separately, the elderly, in addition to those at the highest risk of severe disease and hospitalisation.”

One view is that trial data should be there for all target populations. “If we don't have adequate data in the greater than 65 year old group, then the greater than 65 year old person shouldn't get this vaccine, which would be a shame because they're the ones who are most likely to die from this infection,” said vaccinologist Paul Offit.8 “We have to generate those data,” he said. “I can't see how anybody—the Data and Safety Monitoring Board or the FDA Vaccine Advisory Committee, or FDA decision-makers—would ever allow a vaccine to be recommended for that group without having adequate data.”

“I feel the same way about minorities,” Offit added. “You can't convince minority populations to get this vaccine unless they are represented in these trials. Otherwise, they're going to feel like they're guinea pigs, and understandably so.”