BIAS GAPS IN REAL-WORLD DATA-BASED ECONOMIC EVALUATIONS: UNDERREPORTING, WEAK CONTROL, AND THE NEED TO UPGRADE PROPENSITY SCORE PRACTICE
Author(s)
Yu Xin, Master1, Yilin Deng, Master2, Changjin Wu, PhD3, Yuanyi Cai, PhD4, Jun Hao, PhD5, ling Zuo, Master6, Wen Hui, PhD1.
1Department of Science and Technology, West China Hospital of Sichuan University, Chengdu, China, 2Department of Health Behavior and Social Medicine, West China School of Public Health and West China, Sichuan University, Chengdu, China, 3College of Public Health, Chongqing Medical University, Chongqing, China, 4China Medical University, Shenyang, China, 5Clinical Trial Research Center, The First Affiliated Hospital of Nanchang University, Nanchang, China, 6Integrated Care Management Centre, Outpatient Department, West China Hospital of Sichuan University, Chengdu, China.
1Department of Science and Technology, West China Hospital of Sichuan University, Chengdu, China, 2Department of Health Behavior and Social Medicine, West China School of Public Health and West China, Sichuan University, Chengdu, China, 3College of Public Health, Chongqing Medical University, Chongqing, China, 4China Medical University, Shenyang, China, 5Clinical Trial Research Center, The First Affiliated Hospital of Nanchang University, Nanchang, China, 6Integrated Care Management Centre, Outpatient Department, West China Hospital of Sichuan University, Chengdu, China.
Presentation Documents
OBJECTIVES: Selection bias and confounding are critical factors affecting the validity of observational studies. This study systematically assessed the reporting practices and approaches to controlling such biases in real-world data (RWD)-based economic evaluations (EEs).
METHODS: We searched PubMed for full economic evaluations published between January 1, 2010 and December 31, 2024, in journals ranked within the top 5% by Journal Impact Factor across 51 Clinical Medicine subcategories according to the Journal Citation Reports. Eligible studies were those deriving costs or effects from RWD.
RESULTS: The initial search retrieved 3,320 studies. After title, abstract and full-text articles screening, a total of 82 EEs were ultimately included, encompassing 151 outcome measures from RWD (79 effects, 65 costs, and 7 net benefits). Overall, nearly 15% of studies did not mention any selection bias or confounding at all, while 39.0% failed to implement bias control. Propensity score (PS) analysis was the most commonly used method (31/82, 37.8%), followed by outcome regression models (26/82, 31.7%). Among PS-based analyses, only 41.94% (13/31) of studies reported baseline characteristics for both matched and unmatched cohorts, and merely 46.15% (6/13) used standardized difference to assess covariate balance. Regarding the 151 outcome measures, over 40% lacked appropriate statistical methods for bias control. Net benefit measures had the lowest proportion of missing bias control (14.29%), while gaps for cost and effect measures were similar (44.30% and 44.62%, respectively). Doubly robust estimation following PS adjustment was applied to 31.48% of measures overall, with rates slightly higher for effects (29.62%) than costs (26.09%), but substantially lower than for net benefits (50.00%).
CONCLUSIONS: These results reveal inadequate reporting and control of selection bias and confounding in RWD-based EEs. Although propensity score analysis is widely adopted, the quality of its implementation and reporting requires significant improvement to enhance the credibility of observational economic evaluation.
METHODS: We searched PubMed for full economic evaluations published between January 1, 2010 and December 31, 2024, in journals ranked within the top 5% by Journal Impact Factor across 51 Clinical Medicine subcategories according to the Journal Citation Reports. Eligible studies were those deriving costs or effects from RWD.
RESULTS: The initial search retrieved 3,320 studies. After title, abstract and full-text articles screening, a total of 82 EEs were ultimately included, encompassing 151 outcome measures from RWD (79 effects, 65 costs, and 7 net benefits). Overall, nearly 15% of studies did not mention any selection bias or confounding at all, while 39.0% failed to implement bias control. Propensity score (PS) analysis was the most commonly used method (31/82, 37.8%), followed by outcome regression models (26/82, 31.7%). Among PS-based analyses, only 41.94% (13/31) of studies reported baseline characteristics for both matched and unmatched cohorts, and merely 46.15% (6/13) used standardized difference to assess covariate balance. Regarding the 151 outcome measures, over 40% lacked appropriate statistical methods for bias control. Net benefit measures had the lowest proportion of missing bias control (14.29%), while gaps for cost and effect measures were similar (44.30% and 44.62%, respectively). Doubly robust estimation following PS adjustment was applied to 31.48% of measures overall, with rates slightly higher for effects (29.62%) than costs (26.09%), but substantially lower than for net benefits (50.00%).
CONCLUSIONS: These results reveal inadequate reporting and control of selection bias and confounding in RWD-based EEs. Although propensity score analysis is widely adopted, the quality of its implementation and reporting requires significant improvement to enhance the credibility of observational economic evaluation.
Conference/Value in Health Info
2026-05, ISPOR 2026, Philadelphia, PA, USA
Value in Health, Volume 29, Issue S6
Code
RWD73
Topic
Real World Data & Information Systems
Topic Subcategory
Data Protection, Integrity, & Quality Assurance
Disease
No Additional Disease & Conditions/Specialized Treatment Areas