Validity, Reliability, and Framing Effects in Equity-Efficiency Trade-Off Studies: A Systematic Review

Abstract

Objectives

Health system resources are limited, and decision makers often need to make trade-offs between equity and efficiency. Such trade-offs are guided by public values that are elicited using choice experiments. There is no standard approach to elicit equity-efficiency trade-offs. Previous studies have found significant variability in public values, raising concerns about the robustness of trade-off experiments. The objective of this systematic review was to determine whether and how equity-efficiency trade-off studies consider validity, reliability, and framing effects.

Methods

We searched Medline, EMBASE, and Web of Science in May 2025 for health-related equity-efficiency trade-off studies. Two reviewers independently screened titles and abstracts, followed by full-text review, data extraction, and quality assessment.

Results

122 equity-efficiency trade-off studies were identified, of which 33 studies (27%) investigated validity, reliability, and/or framing effects. Seventeen (13.9%) studies assessed validity, 10 (8.2%) studies assessed reliability, and 11 (9.0%) assessed framing or cognitive effects. Validity was most frequently assessed by comparing results with hypothesized expectations, whereas reliability was commonly assessed by providing a repeated test or questionnaire. Framing and cognitive effects were assessed by varying question order or changing the wording or framing of the scenario. Twenty-three of 27 studies reported high or acceptable validity or reliability, and 6 of 11 studies found no significant framing or cognitive effects.

Conclusions

This article identifies key methodological challenges and considerations that can inform the design of future choice experiments estimating inequity aversion.

Authors

Victoria Chechulina Andrew Chu Quang Hung Lam Camryn Kabir-Bahk Shehzad Ali

Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on ispor.org. Update my browser now

×