OBJECTIVES: Data from randomised clinical trials (RCT) are often considered best evidence for health technology assessments.  The objective of this study was to compare event probabilities used in published cost-effectiveness studies to those observed in actual clinical practice. Selective Cox-2 inhibitors (coxibs) were used as an example. Almost all the 30 published coxib cost-effectiveness studies used RCT data for event probabilities. METHODS: A basic cost-effectiveness model was developed using a decision tree. Two alternative strategies were evaluated: prescription of a conventional NSAID or coxib. The UK General Practice Research Database (GPRD) was used to estimate the individual probabilities of upper gastrointestinal (GI) events during current use of NSAID or coxib. Outcomes included upper GI events as recorded in GPRD and hospitalisation for upper GI events recorded in the national registry of hospitalisations (Hospital Episode Statistics) linked to GPRD. Incremental prescriptions costs were based on GPRD costs. RESULTS: The study population included over 1 million patients prescribed conventional NSAIDs or coxibs. Only a minority of patients used the drugs long-term and daily (34.5% of conventional NSAIDs and 44.4% of coxibs), whereas coxib RCTs required daily use for at least 6-9 months. The rate of upper GI events (as recorded in GPRD) and hospitalisations during current use of conventional NSAIDs decreased over calendar time with 5-8% per year (tests for linear trend P-value <0.05). The mean cost of preventing one upper GI event as recorded in GPRD was £52k (ranging from £32k with long-term daily use to £91k with intermittent use) and £149k for hospitalisations. The mean costs (for GPRD events) over calendar time were £29k during 1990-1993 and £87k during 2002-2005. Using RCT data rather than GPRD data for event probabilities, the mean cost was £8k with the VIGOR RCT and £10k with the CLASS RCT. CONCLUSIONS: The published cost-effectiveness analyses of coxibs lacked external validity and did not represent patients in actual clinical practice. External validity should be an explicit requirement in cost-effectiveness analyses.