OBJECTIVES: NICE was established in 1999 with the remit to reduce variation in the availability and quality of National Health Service (NHS) treatments and care and thus ensure equity in reimbursement decision-making across England. The technology appraisal (TA) process remains at the heart of NICE’s work; if this original aim is to be achieved, the TA process must use a fair and consistent approach. This review aimed to assess elements of the TA process that may lead to inconsistency.

METHODS: Published NICE single technology appraisals (STAs) (excluding withdrawn and terminated) were reviewed on the NICE website up to March 2018. Key TA output data were extracted, including Evidence Review Group (ERG), Appraisal Committee, final recommendation, number of committee meetings, number of weeks from scope to final appraisal determination (FAD), and proportion of decisions going straight to FAD. Descriptive statistics relating to possible sources of variability were calculated.

RESULTS: Up to March 2018, we assessed 267 STAs meeting our inclusion criteria; these were reviewed by 10 ERGs: BMJ (n=23); York (n=29); Aberdeen (n=25); Kleijnen (n=27); Liverpool (n=37); PenTAG (n=16); Sheffield (n=53); Southampton (n=32); Warwick (n=19); and West Midlands (n=6; no longer active). Among ERGs, the percentage of TAs with positive recommendations ranged from 43.8% to 66.7%; percentage optimised ranged from 18.5% to 56.0%; and percentage not recommended ranged from 0% to 18.8%. The number of committee meetings ranged from 1 to 4 (mean, 1.95; median, 2). In total, 69 (25.8%) appraisals went straight to FAD with only one committee meeting.

CONCLUSIONS: Aspects of the review process may result in inconsistency in approach: a previous review identified differences between appraisal committees in terms of appraisal outcome. Although this review found that “recommended” rates differ between ERGs, variability was low with most appraisals resulting in a full or optimised recommendation.