Issues in Evaluating AI-Enabled Healthcare
Moderator
Seamus Kent, MSc, PhD, Erasmus University Rotterdam, Amsterdam, Netherlands
Speakers
Anastasia Chalkidou, London, United Kingdom; Blythe Adamson, MPH, PhD, Doctronic, New York, NY, United States; Ian J Hooley, BS, Pomelo Care, New York, NY, United States
ISSUE: As AI-enabled healthcare technologies pursue reimbursement, a fundamental methodological question confronts evaluators: how should the study population be defined? Should economic evaluation encompass all individuals with access to AI-enabled care, analogous to an intent-to-treat framework, or be restricted to those who actively engage with the technology? This distinction is consequential. An access-based population captures the marginal benefit of reaching individuals who would not have sought care otherwise—people newly engaged through AI-enabled pathways who generate incremental health outcomes. An engagement-based cohort yields more precise effectiveness estimates among users but may understate the technology’s broader value by excluding these marginal gains. The choice directly affects cost-effectiveness conclusions and reimbursement decisions for an emerging class of health technologies.
OVERVIEW: Seamus Kent will moderate. Anastasia Chalkidou (10 minutes) will open with a framing of how the UK and Europe consider the value of clinical AI technologies seeking reimbursement and why population definition matters for evidence generation, drawing on their experience with UK and European HTA processes. Blythe Adamson (15 minutes) will argue for evaluating the broader access-based population, presenting her work on the outcomes and economics of an autonomous AI doctor and demonstrating how economic modeling extends observed clinical outcomes to estimate population health impact, including the value of AI models that expand the reach of healthcare. Ian Hooley (15 minutes) will present his evaluation of the outcomes and economics of a virtual maternity care program at Pomelo Care, advocating for engagement-based cohorts with propensity score methods to achieve balanced treatment populations and produce more credible effectiveness estimates for payers. Twenty minutes will be reserved for audience discussion and debate. This session will benefit health technology assessors, health economists, regulators, AI developers, and payers seeking rigorous evaluation frameworks for AI-enabled care.
Topic
Clinical Outcomes, Health Technology Assessment, Study Approaches