OBJECTIVES:

Clinical effectiveness evidence is a key component of companies’ health technology submissions to the UK National Institute for Health and Care Excellence (NICE) and has to be sought via systematic literature reviews (SLRs). To identify where, and better understand why, NICE Evidence Review Groups (ERGs) have criticised SLRs on clinical effectiveness (clinical SLRs), we reviewed their responses to recent oncology-related submissions.

METHODS:

We searched NICE’s website on 14 April 2022 to identify relevant health technology assessments (HTAs) of oncology drugs published in the previous 12 months. These documents were screened by a single reviewer to identify ERG criticism of clinical SLRs included in submissions. Key critiques were categorised and extracted using a standardised form. Narrative synthesis was conducted to summarise the available data.

RESULTS: We identified 35 oncology HTAs, most of which resulted in positive recommendations from NICE (n=32, 91%). All but one HTA included at least one criticism of the clinical SLR. The most commonly criticised areas were extraction and summarisation of trial data (n=30, 86%), followed by choice and/or application of study-selection criteria (n=22, 63%) and reporting of screening methods and/or results (n=17, 49%). Literature-search methods and quality assessment were less often criticised areas (n=14, 40% and n=9, 26% respectively). The main concerns raised by ERGs included inappropriate selection of population and/or comparator(s); inaccurate or incomplete citation of study results; inadequate search terms use and/or inappropriate search limits; discrepancies between number of included studies and flow-diagram results; and missing or incorrectly conducted quality assessment.

CONCLUSIONS:

Evidence suggests NICE ERGs frequently criticise the methodology and documentation of clinical SLRs included in company submissions. Understanding these criticisms may help to ensure scientific rigour and transparency in conducting and reporting such SLRs, thereby potentially reducing the need to redescribe or revise the research to address queries in ERG clarification letters.