Drug Approval Under Uncertainty Across the COVID-19 Timeline: Cumulative Evidence Versus Value of Information Analyses

OBJECTIVES: The COVID-19 pandemic highlighted the urgent need for effective decision-making in the approval and research of novel treatments. Our study evaluates various decision-making methodologies throughout the stages of evidence accumulation. Possible decisions at each timepoint included rejecting, approving, granting emergency use, or allowing access only in a research setting.

METHODS: We analyzed four decision-making strategies for drug approval and research: the FDA's decisions, cumulative meta-analysis, prospective Value of Information (VOI) analysis (using data available at the decision time), and a reference standard (retrospective VOI analysis using hindsight information). VOI quantitatively determines the benefit of gathering more data by considering the impact on uncertainty, outcomes, and resources. Each time new trial results were published, we assessed the optimal decision per approach and calculated the opportunity losses by comparing the Net Benefit of these decisions to the reference strategy, focusing on each method’s capacity to optimize health outcomes and resource allocation. We used Monoclonal Antibodies (MAbs) as a case study, new trials published between 2020-2021. Our state-transition cohort model simulated hospitalized, ICU-recovered, ward-recovered, or deceased patients, within a US health system-perspective and a $100,000/QALY threshold.

RESULTS: A meta-analysis of 22 studies showed a pooled relative mortality risk with MAbs versus controls of 0.89[95%CI 0.83-0.96, p<0.003]. We found substantial discrepancies between policy decisions and the reference standard, with Net Benefit losses up to $269 billion due to delayed emergency use authorization. Decisions based solely on meta-analysis incurred the highest losses (up to $16 billion), while the prospective VOI approach showed the smallest losses (up to $2 billion).

CONCLUSIONS: Incorporating VOI analysis can improve research prioritization and treatment decisions during pandemics. However, further research is needed to validate the best decision-making model across different situations. Our results offer insights into decision-making strategies in health crises and suggest a framework for future pandemic responses.