Advancing Acceptability of Real-World Evidence for Healthcare Decision Making: A Summary of the ISPOR Real-World Evidence Summit 2024

Madeline Shipley, MPH, ISPOR, Lawrenceville, NJ, USA; Shirley Wang, PhD, Brigham & Women’s Hospital, Harvard Medical School, Boston, MA, USA; Massoud Toussi, PhD, Toussilver, Paris, France; Laura T. Pizzi, PharmD, MPH, Kelly Lenahan, MPH, ISPOR, Lawrenceville, NJ, USA

Since 2017, there have been more than 50 guidance documents published globally on the use of real-world evidence (RWE) in regulatory and health technology assessment (HTA) decision making. The increasing number of guidance documents highlights the importance of RWE to the field of health economics and outcomes research (HEOR). On Sunday, 17 November 2024, ISPOR hosted a Real-World Evidence Summit 2024, a co-located event at the ISPOR Europe 2024 conference in Barcelona, Spain. The Summit covered the latest developments in the use of RWE across the regulatory-HTA-payer decision-making continuum. Four sessions covered methods-related topics, including causal inference and external control arms for comparative effectiveness analyses, the hierarchy of RWE studies, and the role of patient registries. View the full program here. 

Unleashing the Latent Power of RWE in Decision Making

Real-world data (RWD) is defined as “data used for decision making that are not collected in conventional randomized control trials (RCTs).”¹ These data relate to areas such as patient health status and/or healthcare delivery and can come from a variety of sources such as electronic health records, medical claims, surveys, and registries, which can go on to generate RWE. To look more closely at the role of RWE to inform regulatory, payer, and HTA decisions, it is important to showcase success stories where RWE has been pivotal in assessing value and exploring challenges. One such success story example comes from the European Union (EU), where coordinating centers like DARWIN-EU offer a better exchange of and access to healthcare data. Speakers emphasized its usefulness for conducting RWE studies, as it provides cross-national information about prevalence, incidence, treatment patterns, adverse events, and effectiveness, which can be used by national authorities such as the European Medicines Agency and other regulatory bodies conducting HTAs using RWD and RWE. Similarly, agencies like the National Institute for Health and Care Excellence (NICE) have developed their own RWE Framework and routinely use RWE to inform their decisions. NICE has used RWE to inform their reimbursement policies in many ways, including informing the design, parameters, and validation of economic models; understanding the safety of medical technologies; assessing the impact of interventions on service delivery and decisions about care; and assessing the applicability of clinical trials to patients in the National Health Service.

Regulatory and HTA decision makers agree that data quality is an essential component for RWE studies and that a fit-for-purpose RWD source is equally dependent on the question being asked as well as data relevancy and quality. However, there is currently not a clear consensus on how researchers should demonstrate data quality to decision makers. Additionally, as they become more comfortable with RWE to address causal questions, it will become increasingly important to develop guidance and standards for methods that assess the generalizability and transportability of evidence generated from one database to other data and populations. Such analyses will help stakeholders answer the question, “how do these results apply to my patient population?”

Methods for Causal Inference Using RWD

To use RWE in HTA and health policy decision making, it is crucial to use valid estimates of the benefits, harms, and costs of interventions of interest. This requires a closer look at methodological approaches and frameworks for causal inference and their applicability, such as target trial emulation, calibration, and hybrid designs. The target trial in target trial emulation approaches is the ideal hypothetical randomized trial that would answer the causal question of interest. For comparative effectiveness or safety research, this requires the researcher to: (1) ask a well-defined question, and (2) answer the question using observational data (Figure 1).² The target trial emulation estimand is defined by eligibility criteria, treatment strategies, treatment assignment, follow-up, outcome(s), causal contrast(s), and an analysis plan. Emulating these key study design parameters in an observational study may help investigators to clarify the causal study question of interest and avoid design flaws. HEOR professionals should become more familiar with these methods for using RWD to support HTA. External control arms (ECAs) are an alternative study design when RCTs are not feasible or ethical. Like other causal inference methods, it comes with challenges that must be acknowledged, such as selection, measurement, and confounding biases. One example of how measurement challenges can be overcome is by enhancing ECA analyses using calibration and hybrid designs,³ which take primary data from a single-arm trial to identify a larger set of secondary data that is highly similar in baseline characteristics via exact- and or score-matching. Hybrid designs combine underpowered internal control arms, which are arms drawn from the same clinical trial, with external control arms.

Figure 1. Components of the target trial in target trial emulation

Methods like target trial emulation, ECAs, hybrid, and calibrated designs have the potential to support more informed and effective healthcare decision making when used appropriately. Expert use of high-quality RWD for causal inference can help support HTA and policy decision making. However, as with any method, there must still be careful consideration of their strengths, limitations, and applicability to the causal question.

Embracing Diversity and Tackling Heterogeneity in Data, Methods, and Jurisdictions

As decision-maker demand for evidence on the effects of drugs as they are used in clinical practice increases, the need for robust healthcare data infrastructure grows as well. To enhance the value of RWD for decision making, diverse data and methods may be used across various jurisdictions. Multi-database studies are studies that use at least 2 healthcare databases that are not linked to each other at an individual level. Possible sources of heterogeneity in a multi-database study can include variation in study protocols, variation in study quality, differences in interventions, differences in measurement or data capture, differences in follow-up length, and treatment-covariate interaction. But heterogeneity is not inherently negative. Possible sources of heterogeneity that are inherent to the data, such as measurement differences, are unwanted, but clinical heterogeneity, such as differences in prescription patterns between geographical regions, can be informative (Figure 2).⁴ Unwanted heterogeneity can be mitigated through harmonization of protocols or data. One way that DARWIN-EU manages the inevitable heterogeneity of routinely collected data is by converting disparate source data into the Observational Medical Outcomes Partnership Common Data Model, which uses a common structured format and ontologies.

Figure 2. Visual representation of homogeneity versus heterogeneity

Overcoming Obstacles: Charting the Path Ahead

Multistakeholder learning networks such as RWE4Decisions have focused on identifying the gaps in RWE that need to be filled to inform HTA/payer decisions. By promoting collaboration across 7 different stakeholder groups, which consists of national HTA/payer collaborations, pharmaceutical industry, clinical teams, patient groups, disease registry holders, and RWD analytics experts, learning networks like RWE4Decisions helps those stakeholders generate and utilize better RWE to inform HTA, joint clinical assessments, and payers. The 58 actions for stakeholders fall under 1 of 4 pillars supporting development of robust RWE: (1) data availability, governance, and quality; (2) methodology design and analysis; (3) policy and partnerships; and (4) trust and transparency (Figure 3).⁵

Figure 3. RWE4Decisions 4 pillars to support development of RWE

Abbreviation: RWE, real-world evidence

The ALIGN Matrix—aligning investigational designs and data sources with evidence needs in healthcare research—has been proposed as a tool to address the complexity of matching evidence needs with appropriate study designs and data sources. Ensuring that data sources are aligned with evidence needs when collecting RWD is beneficial to a researcher as it will allow the collection of quality data that matches their needs.

In conclusion, the Real-World Evidence Summit underscored the growing impact of RWE in shaping healthcare decision making. Advancements in methods like hybrid designs and target trial emulation, as well as collaborative initiatives such as DARWIN-EU and RWE4Decisions, are helping to address challenges in areas like data quality and reliability, transportability, and applicability. Continued collaboration to meet decision-making needs will be key to unlocking the full power and potential of RWE. ISPOR is actively working to improve standards and practices for improving the quality of RWD to increase the generation of rigorous RWE and to promote transparency in conduct and reporting of RWE studies. Additional details about RWE initiatives at ISPOR are available here. ISPOR will host its next Real-World Evidence Summit—Through the Lens of Asia Pacific—28-30 September 2025 in Tokyo, Japan. More information is available here.

References:

1. Garrison LP, Neumann PJ, Erickson P, Marshall D, Mullins CD. Using real-world data for coverage and payment decisions: The ISPOR Real-World Data task force report. Value Health. 2007;10(5):326-335. doi:10.1111/j.1524-4733.2007.00186.x
2. Dickerman B. From healthcare data to decisions: the target trial framework. Presented at: ISPOR Real-World Evidence Summit 2024; November 17, 2024; Barcelona, Spain.
3. Schneeweiss S. Enhancing external control arm analyses through data calibration and hybrid designs. Clini Pharmacol Ther. 2024;116(5):1168-1173 doi:10.1002/cpt.3364
4. Quinten C. Possible sources of heterogeneity and how this can be assessed. Presented at: ISPOR Real-World Evidence Summit 2024; November 17, 2024; Barcelona, Spain.
5. Rannanheimo P. Stakeholder actions to generate better real-world evidence for HTA/payer decisions. Presented at: ISPOR Real-World Evidence Summit 2024; November 17, 2024; Barcelona, Spain.