Understanding the Potential Threat of Unmeasured Confounding in RWE Studies: What Statistical Methods Can Be Used for Robustness Assessment?

OBJECTIVES: Quality of evidence and associated uncertainty will be critical aspects of joint clinical assessment [EunetHTA21 D4.5]. In recent months, Canada’s drug agency has requested robustness checks to alleviate concerns about uncertainty due to unmeasured confounding in RWE studies. Currently, there is lack of accessible guidance on which methods to use for sensitivity analysis. This research aimed to provide practical considerations that will aid sponsors in selecting appropriate methods for assessing the impact of unmeasured confounding.

METHODS: We conducted a literature search to identify reviews of methodology for quantifying the impact of unmeasured confounding in observational studies. Analytical methods were captured, and frequency of their use in the literature was determined. Strengths and limitations of the most applied methods were documented. Alternative approaches were evaluated based on their promise to fulfil important limitations of the established methods. We found methods to cover a range of scenarios that would be encountered in the HTA setting.

RESULTS: The literature search identified eight overviews of methods for evaluating potential impact of unmeasured confounding. Simple deterministic approaches were identified as most applied; use of the E-value was high, followed by the Rosenbaum-Rubin sensitivity analysis. Negative controls were frequently used to address multiple bias sources. Alternative methods, including probabilistic modelling approaches, were less frequent but offer ability to incorporate what is known about suspected unmeasured confounders.

CONCLUSIONS: We observed high use of simple and convenient methods to address concerns about unmeasured confounding. Given the short time frames for re-submission, the same can be expected in response to requests for sensitivity analysis from HTA agencies. A framework is needed to aid pre-selection of methods, enabling sponsors to include sensitivity analysis results within HTA submissions. By providing practical considerations, we build the framework, with the goal of leading to more informed and timely decision making.