OBJECTIVES: The win ratio (WR) measures the benefit of a treatment across a series of endpoints ordered based on clinical importance, expressing the effect as the odds of a better outcome with the experimental treatment. The WR can enhance clinical insights but its utility as an effect measure in health technology assessment is unclear, particularly whether reliable indirect treatment comparison (ITC) of WRs is possible with conventional methods that rely on an assumption of (conditional) consistency. We investigated this in a simulation study.

METHODS: Responses for two outcomes in trials comparing A to B and C to B were simulated using R. Responses with A and B were fixed and the effect of C was varied relative to A (i.e., more, less or similarly effective as A). WRs were derived comparing A-B and C-B. The ratio of these was taken as an indirect estimate of the WR for A-C and compared to a the WR calculated directly from patient level data for A and C. The percent difference between the direct and indirect estimates was examined across replications to assess the presence and extent of bias.

RESULTS: Simulations showed that the accuracy of the ITC-derived WR varied with the relative effect sizes of the treatments. When C was more effective than A and B, the ITC-WR underestimated the direct WR. The magnitude of bias became larger when the effect of C vs. B was larger than A vs. B. The bias was small and potentially ignorable (< +/-5% on average) when the effect of C was comparable to A or B or when C was less effective than B.

CONCLUSIONS: ITC of WR with conventional methods based on aggregate data and is prone to bias. Alternative approaches that do not require the consistency assumption should be explored given the increasing use of the WR.