The United Kingdom National Institute of Health Care Excellence (NICE) published a draft clinical guideline for consultation on anticoagulation therapy for stroke prevention in individuals with atrial fibrillation in September 2020 that addressed which non-vitamin K antagonist oral anticoagulant (NOAC) therapy (apixaban, dabigatran, rivaroxaban, edoxaban) is most clinically and cost-effective. We aimed to elucidate methodological considerations and challenges involved in evaluating the comparative effectiveness and cost-effectiveness of within-class treatments for the purpose of decision making by a reimbursement authority. To better understand the differences across NOACs in terms of comparative and cost-effectiveness, we performed fixed effect and random effects meta-analyses and produced corresponding forest plots using clinical outcomes from a competing risk network meta-analysis and data from cost-effectiveness analyses (i.e., quality adjusted life year [QALY] estimates and net monetary benefit [NMB] values) reported in the evidence review. Heterogeneity across treatments was reported as I² values and Q-statistic values. The base case cost-effectiveness plane and cost-effectiveness acceptability curve (CEAC) of the review were assessed in terms of ability to support decision making for within-class treatments. Forest plots of the hazard ratios of seven key clinical outcomes suggested that no single NOAC outperformed the others across all outcomes. For the seven outcomes, the I² values ranged from indicating no heterogeneity to substantial heterogeneity across treatments. The I² values for the QALY and NMB outcomes suggested no differentiation between therapies. The cost-effectiveness plane demonstrated similar cost-effectiveness among NOACs (at NHS List prices); however, the CEAC provided a misleading view of cost effectiveness. Graphical methods, such as forest plots and cost-effectiveness planes, help highlight the challenge of distinguishing a treatment option that clearly outperforms within class. The recent NICE consultation indicated the possible confusions from relying on CEACs with many competing options as an area of concern for concealing uncertainty in the ranking process.