OBJECTIVES: Payers are seeking improvements in outcomes that are meaningful for the patient, but the preference of payers on what change can be considered meaningful is not well defined. Clinically relevant differences (CRDs) in outcomes and grading of their magnitude in oncology are being established by both European and American oncology organisations (European Society for Medical Oncology (ESMO) and American Society of Clinical Oncology (ASCO)). This indicates a transition from a focus primarily on statistical significant improvements (i.e., “is there a difference?”) in outcomes towards the clinical relevance of these improvements (i.e., “does the difference matter to patients?”). The attitude of payers towards CRDs in oncology outcomes is not well-understood, with little guidance around oncology CRDs from payers. The objective of this study is to evaluate the alignment between payers and clinicians in their assessment of clinical and health benefit of oncology products. METHODS: Oncology products launched recently were evaluated using the approach suggested by ESMO and ASCO. For the same products, the payer decision was evaluated to establish the clinical and health benefit rating by NICE (UK), HAS (France) and G-BA (Germany). RESULTS: Not all products granted market approval have been evaluated by payers. The research showed that where they had been evaluated, payer quantification of clinical benefit differed to that recommended by oncology societies. Furthermore, clinical benefit assessment, particularly regarding overall survival improvement, differed between payers themselves. CONCLUSIONS: Oncology societies are recognising the need to ensure consistent assessment and representation of clinical benefit of new oncology products. Whilst payers often have guidance on how they assess benefit, this is often generic and applied across therapy areas. As a consequence, there still remains an inconsistent approach to evaluating clinical benefits in oncology between payers, which provides challenges and implications in drug development programmes for novel oncology therapies.