TECHNICAL ERRORS IN COST-EFFECTIVENESS MODELS- EVIDENCE FROM THE SINGLE TECHNOLOGY APPRAISAL PROGRAMME IN ENGLAND AND WALES

Author(s)

Trueman D*, Livings C Abacus International, Bicester, United Kingdom

OBJECTIVES: Modelling for cost-effectiveness studies often relies upon the use of spreadsheets. However, research has shown that approximately 90% of spreadsheets contain technical errors. Furthermore, cost-effectiveness models rely on accurate transcription between many data sources, which increases the risk of errors further. The objective of this analysis was to ascertain the incidence of reported technical errors in cost-effectiveness models submitted to NICE as part of the Single Technology Assessment (STA) programme, which are subject to rigorous assessment by Evidence Review Groups (ERGs). METHODS: NICE guidance documents were searched for a wide range of technical error types using the HTAinsite database. Reports were included if the ERG had identified technical errors in the manufacturer’s submission and this had been noted at committee level. Included appraisals were analysed to identify categories of errors identified. RESULTS: Of the 102 completed STA Guidance documents searched, 39 appraisals met the inclusion criteria of the study, representing a technical error incidence of 38.2% (95% CI: 28.8 – 48.4%). Within these studies, 47 errors were identified in the following areas: computation (47%), logic (17%), transcription (13%) and data handling (9%). Error causes could not be determined in 15% of cases. The magnitude of effect caused by technical errors was difficult to determine, because corrected models often include additional changes to parameters or model structure. CONCLUSIONS: The incidence of technical errors identified in the STA programme was lower than has previously been observed in studies of spreadsheet validity although this analysis assumes that ERG groups will identify all technical errors. The true incidence of errors may be higher than reported by this analysis. Use of best-practice methods and increased awareness of the causes and identification of technical errors may help to reduce their pervasiveness.

Conference/Value in Health Info

2013-11, ISPOR Europe 2013, The Convention Centre Dublin

Value in Health, Vol. 16, No. 7 (November 2013)

Code

PRM97

Topic

Methodological & Statistical Research

Topic Subcategory

Modeling and simulation

Disease

Multiple Diseases

Explore Related HEOR by Topic


Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on ispor.org. Update my browser now

×