When Will We Have a COVID-19 Vaccine? Predictions, Analysis, and Questions Answered

Today we launch a new paper, COVID-19 Vaccine Predictions, that uses mathematical modelling and expert interviews to learn more about COVID-19 vaccine portfolio and generate probabilistic estimates on when we will have a safe and efficacious vaccine and how long will it take for manufacturing scale up to produce sufficient doses. This is part of a body of work that will include a simple-to-use web tool that will generate vaccine predictions, to be launched in November, and a series of virtual events.

Here we discuss our research, outlining what it does and does not do, and note its limitations.

What are we trying to achieve?

While the world waits for a COVID-19 vaccine, discussions continue about what that vaccine might look like and the need to diversify vaccine research and development portfolios. We wanted to add a systematic and evidence-informed approach that could estimate timelines for vaccine development, approval, and manufacturing. In doing so we highlight the importance of portfolio diversification.

Until clinical trials are conducted, no one knows which vaccines will work or when, or even if, any of the COVID-19 vaccines currently under development will be approved. Nonetheless, we believe it is possible to come up with useful objective estimates.

We interviewed a panel of 16 experts from governments, academia, and industry, and collated information on COVID-19 vaccine candidates and their attributes, and how important each of these are to getting to a successful (licensed) vaccine. We built a mathematical model that takes a range of inputs on different types of vaccines and estimated timelines for each of the COVID-19 vaccines. Our results include ranges in the estimated probabilities to capture the significant underlying uncertainty. We’ve run these inputs through our model, and explain our results in full in the paper. Our code is publicly available, and we look forward to launching our vaccine prediction tool in November.

As with many things with COVID-19 or any new pathogen, we are operating with limited information.

What is our core message?

Whilst our report describes in detail our methods and findings on when a vaccine is likely to be available for distribution, our core message is less about the “when” (which is an estimate surrounded by considerable uncertainty), and more about the value of portfolio diversification. Diversification will maximise the chances of getting to a vaccine that works as soon as possible.

We also conclude, as have others before us, that while waiting for a vaccine, the world must learn to live with the virus, and practice various nonpharmaceutical interventions such as social distancing. We should take a holistic approach that also aims to reduce the impact of policy measures on health and livelihoods beyond COVID-19 (such as the indirect health impacts, as our colleagues have explored).

Is this approach common?

In practice, probabilities of success for new therapeutics and vaccines are usually estimated using a combination of past data and expert judgement.

Given that some of the COVID-19 vaccine platforms are novel, and that understanding of basic biology and immune protection for this new pathogen is evolving, past data do not tell us as much. So, we used past data for each stage of success to create our baseline estimate, which we provided to each of the experts and then asked them to update it with their subjective knowledge. Whilst our methodology is robust and standard practice in the field, we acknowledge that our sample size is not very large, constrained by the limited number of experts in this specific field and by our aim to produce something timely which can be of use in current planning.

We built on work by Vladimir Shnaydman et al., who used Monte Carlo (MC) simulation to understand COVID-19 vaccines. Like their work, our model is probably slightly more complex than other therapeutic areas. Because of the large number of candidates and intertwined relationships, we also use MC simulations for estimating when a vaccine is likely to be approved.

By making our model available, we hope that we and others can update the inputs as more information becomes available and overall uncertainty is reduced. We also acknowledge that different models or approaches will estimate different POS. Such differences in POS using different methodology, different input data, and different assumptions are very much in the spirit of advancing science through different ways of estimating uncertain events and of validating and interpreting them. We look forward to discussing how our methods and inputs can be improved.

On production capacity, we have built on analyses by the Coalition for Epidemic Preparedness Innovations (CEPI) and others, and worked with a range of manufacturing experts and biochemical engineers. Again, there may be differences in the approach and end results with some national leaders and coalitions coming up more aggressive estimates than ours especially when thinking about individual countries.

How does our model work?

Monte Carlo simulations apply probabilistic rules repeatedly, to simulate many future outcomes, and understand how likely something is to happen.

Our research and development model aims to simulate the COVID-19 vaccine portfolio by using probabilities of success given to us by experts. In each simulation, vaccines succeed or fail in line with the model’s inputs. Because in the real-world vaccines based on similar technologies are likely to succeed or fail for the same reason, when one vaccine is approved by the model or fails, we adjust the probability of success for other vaccines. To develop this model, we worked with high energy physicists who also use MC simulations in their quest for evasive particles such as the Higgs boson.

This MC simulation feeds into a model built with Bryden Wood that estimates how long it will take to get factories ready to start producing vaccines as well as global capacity to manufacture different types of vaccines. This uses the output from each MC simulation and projects how long we would expect it would take to produce enough vaccines for different target groups as identified by the WHO: healthcare professionals, individuals over 65, those with co-morbidities, and the rest of the world. We do not look at distribution and vaccine uptake nor do we model scenarios for reaching herd immunity or for minimising deaths or revising the economy by returning people to work.

When does our model say we might have vaccines ready for distribution?

Using the inputs from our experts and background information, the model suggests that there is a 50 percent chance that by the end of April 2021 there will be at least one vaccine safe and efficacious enough to win approval from a stringent regulator as defined by WHO. By the end of 2021, this rises to 85 percent. When we feed these results into the manufacturing model, the results demonstrate it will probably take more than a year to produce enough doses to vaccinate healthcare professionals globally, and that it could be September 2023 before we have enough doses to cover the global population. We may well need fewer doses to reach herd immunity globally, and the timelines within each country are likely to be very different too given the number of bilateral deals and distinct country coalitions.

Figure 1. Projected probability that at least one COVID-19 vaccine is approved, October 2020–September 2023

Image

In our analysis, we only estimate how long it will take to develop and manufacture a vaccine. We do not include any considerations regarding financing the purchasing of the vaccine, building the infrastructure for distributing and administering across countries, or, most importantly, convincing people to be vaccinated. Nor do we look at optimal allocation scenarios within and across countries to get to herd immunity sooner, reducing overall number of deaths or getting the economy back up and running.

In this sense, our estimates need to be combined with complementary analyses to come up with estimates of when things will return to the pre-COVID-19 “normal.”

How confident are we that our model is right?

When examining a complex situation like this, it is never possible to build a model that is as complex as real life and can predict the future with certainty. And the more complex a model, the more likely it is to contain one or more errors. For this reason, modellers always make assumptions that simplify processes. For example, in our model, we only vary vaccines’ probabilities of success based on which broad type of underlying technology they use, which one of five funding categories they are in, and which stage of clinical development they are in. Therefore, there are many vaccines that are in the same funding category and stage of development that our model treats as being identical, even though in practice these might be quite different, and there may be reasons to believe one is more promising than another. If we have done our job correctly, the assumptions that we use will not stop the model from being broadly right, but it will never be perfectly accurate.

Furthermore, our results are based on the expert interviews we collected. As well as modelling the average response from our experts, we also modelled what would happen if very optimistic and pessimistic respondent inputs were used. The optimistic inputs suggest near certainty that we’ll get a vaccine approved in 2021, with January being the month that approval is most likely, the pessimistic inputs suggest that it will be April 2022 before it becomes more likely than not that a vaccine is approved.

Are there things we could do better to develop and manufacture an effective vaccine sooner?

Mostly high-income country governments, likely the major purchasers of a future vaccine, and coalitions such as CEPI/Gavi Covax Facility, have tried to diversify their portfolio to include several different candidates, so that if one candidate fails hopefully another will succeed. Our R&D modelling suggests that they have done a good job of building a portfolio aimed at getting a vaccine approved quickly. At the same time, governments have invested in manufacturing scale up at risk to fast track production of the early successful candidates. Our manufacturing modelling suggests that there is significant capacity in the world to manufacture the successful products but flags that, despite existing global capacity for manufacturing inactivated viral vaccines, only one of the 172 candidates being developed outside of Asia is an “inactivated virus.” Linking PoS with manufacturing processes and bottlenecks is a valuable aspect of our approach.

Overall, a message from our earlier work supported by this analysis, is that countries should share risk more both through global (ideally) efforts which include including middle-income countries (MICs) and encourage the private sector to invest, on an ongoing basis, in R&D. Purchasing agreements and manufacturing at risk may speed up the development process but perhaps they offer less of an incentive for diverse players to enter the market to the extent most payers are already locked into deals for purchasing specific but yet to be developed and assessed products. Further, so-called vaccine nationalism creates multiple national border driven silos which again undermine the chances of succeeding. Many countries have leading vaccine candidates, and it is unclear which ones will succeed. Making agreements now, to share vaccines with other countries if domestic candidates are successful, in return for receiving vaccines from foreign countries if they are not, is a win-win deal. Manufacturing bottlenecks could also be addressed through ex ante multicounty agreements. Such agreements however require reciprocity, and policy makers to understand that it may be in the country’s longer term interest to export their successful vaccine before vaccinating their whole population, if they have managed to approve one before other countries; but these are difficult decisions politically.

Once enough vaccines are manufactured, will life return to normal?

This is a question we did not set out to answer. What we do know is that we will most probably not get a single vaccine which completely blocks transmission and works effectively across all age groups with minimal or no side effects and which the vast majority of the population of the world has access to and agrees to take. Returning to normal is a function of different factors, including vaccine efficacy and mode of work, coverage rates, the point at the epidemic when the vaccine is introduced, and how long it offers protection, amongst other factors.

One of the questions we asked our experts was how efficacious they expected the first vaccines to be. Most told us that it was very hard to know the exact efficacy without seeing phase 3 trials. However, when asked for a quantitative estimate, the average response was just over 60 percent reduction in disease incidence, which they thought was probably too low to eradicate the illness as a single measure. Instead respondents thought that second or third generation vaccines, which are formulated and improved using the information we have gleaned from the current portfolio, will be much more likely to give the 90 percent+ efficacy that we have often come to expect of vaccines. Many also highlighted the fact that we are not sure how long a vaccine’s protection would last. You can find more information on this and other qualitative responses here.

Further, vaccination is not the only route out of this crisis. Tremendous progress is being made on rapid diagnostics, and important studies are taking place to find better treatments for people who are ill with COVID-19. We have not undertaken research into R&D in these areas, but governments should be investing there, in case our pessimistic experts are right and it is mid-2022 before we start seeing vaccines getting approved.

--

CGD undertook this project in partnership with Ariadne Labs. If you would like to know more about this work, you can read our main report, Bryden Wood’s report on manufacturing COVID-19 vaccines, and a blog that we have written outlining the qualitative information we received from the interview process.

This work was undertaken in collaboration with:

Robert Van Exan; President, Immunization Policy, and Knowledge Translation. A vaccine expert with 35 years' experience at Sanofi Pasteur,

Steve Lloyd; Emeritus Professor, School of Physics and Astronomy, Queen Mary University of London,

Laura Subramanian; Senior Specialist, Monitoring and Evaluation, Ariadne Labs,

Adrian La Porta; Technical Director, Bryden Wood,

Jiabin Li; Process analyst, Bryden Wood,

Eddine Maiza; has 25 years of experience in the pharmaceutical industry including R&D Portfolio Management and New Product Planning, he contributed to this article in his own personal capacity,

David Reader; Process director, Bryden Wood,

Julie Rosenberg; Assistant Director, Better Evidence, Ariadne Labs,

Jack Scannell; Innogen Associate & Honorary Fellow School of Social and Political Science University of Edinburgh and JW Scannell Analytics LTD,

Vaughan Thomas; Tillingbourne Consulting, RAEng Visiting Professor at UCL,

Rebecca Weintraub; Director, Better Evidence at Ariadne Labs and Faculty, Global Health and Social Medicine, Harvard Medical School, and Associate Physician, Brigham and Women's Hospital,

This report was independently reviewed by four peer reviewers, to sense check assumptions and our approach, thought the authors are fully responsible for any assumptions or errors.