Objectives:  To develop a set of criteria to critically appraise the strengths and weaknesses of health economic methods for the systematic valuation of interventions for ultra-rare disorders (URDs). Methods:  An international group of clinical and health economic experts met in conjunction with the Annual European ISPOR Congresses in Berlin/Germany and Dublin/Ireland, November 2012 and 2013, to deliberate and agree on a set of criteria to assess the potential of the various methods, which have been used or proposed to estimate the social value of medical interventions for URDs. Results:  The group identified a broad set of potential criteria, which may be grouped according to the following dimensions:  theoretical foundations (normative premises, i.e., links to moral and economic theories, including - but not limited to – nonutilitarian consequentialist and deontological reasoning, definition and treatment of core concepts of economic thinking such as opportunity costs and efficiency), empirical underpinnings (social preferences related to attributes of the health condition or of the person afflicted with it), and pragmatic aspects (feasibility of implementation and potential for bias and misuse).  For each of the dimensions, a set of criteria has been agreed upon, which in turn will need further scrutiny and justification.  Conclusions:  Previously, a need had been identified for modifications or alternatives to the conventional logic of cost effectiveness applying benchmarks for the maximum allowable cost per quality-adjusted life year (QALY).  We propose a framework for the systematic assessment how well different evaluation approaches reflect prevalent social norms and value judgments.  As a next step, the framework shall be applied on multi-criteria decision analysis methods and social cost value analysis, either using the person trade-off (PTO) or the relative social willingness-to-pay (RS-WTP) instrument.