Advancing Value of Information Methods: Does Conducting Further Research Directly Translate Into Changes in Clinical Practice, or Do Implementation Rates of Interventions Depend on the Levels and Types of Information Generated?

OBJECTIVES: Value of information (VOI) analyses can inform resource allocation decisions regarding research and implementation activities. It can also establish efficient study designs prior to investigation.

Current VOI methods rely on assumptions that there is no relationship between evidence generation and implementation efforts, or that the relationship is positive. However, this dynamic relationship has not been empirically evaluated. In this exploratory study, we aimed to investigate the impact of different levels (parameter uncertainty) and types (study outcomes) of information, on implementation rates.

METHODS: Diffusion of innovation theory informed our understanding of how implementation rates may be expected to differ by parameter uncertainty, study outcomes, and study setting. We built on previous research using expert elicitation and demonstrate how real-word evidence and trial data can be used to investigate the dynamic nature of implementation rates. We then explored the relationship between different levels and types of information and implementation rates in two pragmatic trials—a COVID-19 adaptive platform trial and a non-COVID-19 randomized controlled trial.

RESULTS: Implementation rates of effective therapies and de-implementation rates of harmful and futile therapies can be affected by the strength and quality of existing and new evidence, outcome measures, investigative products, study design, costs and resource availability, and the extent to which change was practical and feasible. We recognize that the COVID-19 pandemic created a unique environment where implementation was rapid in the absence of prior knowledge but barriers to implementation persisted.

CONCLUSIONS: VOI analyses must account for the interplay between information (parameter uncertainty and study outcomes) and implementation to accurately inform resource allocation decisions. Real-world evidence and trial data can be used to investigate the potential impact of different levels and types of information on implementation rates. This work creates a foundation for trial design optimization methods and further advances VOI methodology.