Objectives: The Federal Joint Committee (GBA) and the Institute for Quality and Efficiency in Health Care (IQWiG) in Germany as well as the National Institute for Health and Care Excellence (NICE) in England are broadly considered as HTA agencies following robust and transparent assessment procedures. The objective of our study was to test the impact of official evaluation criteria on HTA outcomes and, in addition, we analyzed the potential effect of the budget impact.

Methods: We extracted data from all publicly available GBA decisions and IQWiG assessments as well as NICE single technology appraisals (STAs) between January 2011 – when early benefit assessments (EBAs) were implemented in Germany – and June 2018, as well as all relevant information with regard to evaluation criteria used by the HTA agencies. The regression analysis consists of an ordered logit model that estimates the effect of the budget impact on the HTA outcome(s) and that controls for the evaluation criteria used by GBA/IQWiG and NICE.

Results: Overall, we considered 262 completed EBAs for GBA/IQWiG and 207 STAs for NICE. NICE recommendations are largely driven by the incremental cost-effectiveness ratio and, if applicable, by the end-of-life criteria (p<.01). IQWiG assessments are significantly affected by the availability of randomized-controlled trials and patient-relevant endpoints (p<.01), whereas GBA appraisals primarily focused on endpoints (p<.01). Interestingly, the budget impact correlated with NICE STAs (inverted-U relationship, p<.1) and IQWiG assessments (increasing linear relationship, p<.05), but not with GBA decisions (p>.1). However, as IQWiG assessments seem to be more rigorous than GBA appraisals regarding the consideration of evaluation criteria, decisions by GBA might be negatively associated with the budget impact.

Conclusions: Regression results indicate that GBA/IQWiG and NICE follow their official evaluation criteria consistently. After controlling for all significant variables, the budget impact seems to have an effect on HTA recommendations.