Health Technology Decision-Making in an Imperfect World: A Unifying Framework of Value of Information and Implementation Metrics Based on a Narrative Review

OBJECTIVES: Value of information (VOI) analyses can support research prioritization and design as they calculate the value of reducing uncertainty in decision-making. Whilst population levels are commonly adjusted by implementation, individual VOI analysis implicitly assumes that future optimal interventions are fully implemented. This unrealistic assumption has driven the development of methods that consider the value of implementation alongside the value of information. However, these methods are unevenly developed and rarely used in practice. Our objective was to review value of information and implementation concepts and develop a unifying taxonomy, aiming to improve the use of these methods going forward.

METHODS: We performed a review of value of implementation methods and clarified their interpretation using a toy example. We clearly identified when these measures compute the value of undertaking future research to reduce uncertainty and when they estimated their value of improving implementation. We developed a single unifying framework and illustrated it in the toy example.

RESULTS: The unifying framework consists of seven unique concepts that can be categorized as individual value of information / implementation measures that make one of the following assumptions: (i) the optimal intervention is perfectly implemented (i.e. traditional VOI measures), (ii) no further information is collected or that it does not have an effect on uncertainty reduction (traditional value of implementation measures), and (iii) information and implementation levels both change, which requires that the strength of evidence is directly linked to imperfect implementation levels.

CONCLUSIONS: We clarified the interplay between value of information and value of implementation methods. Characterizing the relationship between the strength of evidence and implementation levels is challenging but crucial to ensure accurate research prioritization. We hope that the clarifications provided here, combined with further methodological research, will improve the prioritization of research and implementation strategies in an imperfect world.