TECHNICAL ERRORS IN COST-EFFECTIVENESS MODELS- EVIDENCE FROM THE SINGLE TECHNOLOGY APPRAISAL PROGRAMME IN ENGLAND AND WALES
Author(s)
Trueman D*, Livings C Abacus International, Bicester, United Kingdom
Presentation Documents
OBJECTIVES: Modelling for cost-effectiveness studies often relies upon the use of spreadsheets. However, research has shown that approximately 90% of spreadsheets contain technical errors. Furthermore, cost-effectiveness models rely on accurate transcription between many data sources, which increases the risk of errors further. The objective of this analysis was to ascertain the incidence of reported technical errors in cost-effectiveness models submitted to NICE as part of the Single Technology Assessment (STA) programme, which are subject to rigorous assessment by Evidence Review Groups (ERGs). METHODS: NICE guidance documents were searched for a wide range of technical error types using the HTAinsite database. Reports were included if the ERG had identified technical errors in the manufacturer’s submission and this had been noted at committee level. Included appraisals were analysed to identify categories of errors identified. RESULTS: Of the 102 completed STA Guidance documents searched, 39 appraisals met the inclusion criteria of the study, representing a technical error incidence of 38.2% (95% CI: 28.8 – 48.4%). Within these studies, 47 errors were identified in the following areas: computation (47%), logic (17%), transcription (13%) and data handling (9%). Error causes could not be determined in 15% of cases. The magnitude of effect caused by technical errors was difficult to determine, because corrected models often include additional changes to parameters or model structure. CONCLUSIONS: The incidence of technical errors identified in the STA programme was lower than has previously been observed in studies of spreadsheet validity although this analysis assumes that ERG groups will identify all technical errors. The true incidence of errors may be higher than reported by this analysis. Use of best-practice methods and increased awareness of the causes and identification of technical errors may help to reduce their pervasiveness.
Conference/Value in Health Info
2013-11, ISPOR Europe 2013, The Convention Centre Dublin
Value in Health, Vol. 16, No. 7 (November 2013)
Code
PRM97
Topic
Methodological & Statistical Research
Topic Subcategory
Modeling and simulation
Disease
Multiple Diseases