Presentation (CTI)

OBJECTIVES: Article 31 requires the European Commission to evaluate the EU HTA framework by 2028 — yet the Regulation never defines what "success" looks like or how to measure it. This study developed an integrated evaluation framework, linking the Regulation's objectives to diagnostic criteria and measurable indicators, to support an evidence-based first Article 31 review.
METHODS: The Regulation's objectives were extracted from its text and recitals and organised into a structured taxonomy. These were mapped to the OECD-DAC evaluation criteria — relevance, coherence, effectiveness, efficiency, impact, and sustainability — and to a Key Performance Indicator (KPI) set classified by structure, process, output, outcome, and system impact. Indicators were assessed for feasibility, measurability, and policy relevance, drawing on health-system performance literature and established KPI methodologies.
RESULTS: The framework distinguished what to evaluate from how to measure it. At the diagnostic level, the DAC criteria translated broad objectives into reproducible evaluation domains; several objectives — reduced duplication, improved cooperation, procedural performance — proved directly measurable, while others — patient access, innovation incentives, sustainability — depend on national factors and resist attribution. At the measurement level, a balanced KPI portfolio combined efficiency (timelines, resource use), effectiveness (completion, methodological consistency, participation), outcome (duplication reduction, Member State uptake), and system-impact (access, evidence generation, innovation) indicators. A consistent theme was the gap between activity and outcome: counting completed assessments measures output, not success. The analysis also surfaced topics under-anticipated at adoption, including methodological convergence and evidence-generation burden.
CONCLUSIONS: Counting assessments is not the same as measuring success. Agreeing evaluation domains and a balanced indicator set before 2028 — while acknowledging the limits of attribution — would give the first Article 31 review focus, credibility, and an evidence base for reform.