Presentation (CTI)

OBJECTIVES: To quantify associations between appraisal structural characteristics and the presence, direction, and decision-relevance of explicitly documented methodological critique in NICE technology appraisals.
METHODS: All publicly available NICE technology appraisal committee papers published between 01/01/2025 and 31/12/2025 were reviewed using a structured AI-assisted extraction protocol. For each paper, metadata and ten pre-specified binary critique domains were extracted. This yielded 114 appraisals. The primary analysis was restricted to first submissions (n=46) to preserve independence. Domains demonstrating near-universal results was excluded, and a Core Issues Score (sum of remaining domains) was used as the primary continuous outcome. Associations were assessed using χ²/Fisher’s tests with bias-corrected Cramér’s V and Mann-Whitney/Kruskal-Wallis tests with effect sizes. Odds ratios were estimated for each critique domain versus recommendation outcome to assess directionality. False discovery rate correction and minimum effect-size thresholds were applied. A supplementary analysis evaluated whether model-type composition influenced observed associations.
RESULTS: Eighteen significant associations (q<0.05, effect ≥ threshold) were identified and directionally replicated in the full dataset. Appraisal type and model type were co-determined (V=0.545). Issues in comparative treatment effect estimation across studies showed the strongest associations with overall critique burden (rank-biserial r=0.956 and 0.763) and were associated with reduced odds of recommendation. While utility and model structure critique were strongly associated with recommendation outcomes in univariable analyses (V=0.663-0.688), supplementary analyses demonstrated that these effects were largely driven by cost-comparison models, which exhibited higher recommendation rates and fewer recorded utility and structural issues due to model simplicity.
CONCLUSIONS: Methodological critique in NICE appraisals in 2025 is systematically structured and reflects both underlying model complexity and appraisal context. Evidence synthesis complexity, particularly indirect comparisons (ITCs), is a key driver of critique burden and reduced recommendation likelihood. These findings identify ITC and evidence synthesis quality as the highest-priority targets for pre-submission investment.