OBJECTIVES: The ‘evidence hierarchy’ of randomized controlled trial (RCT) over non-RCT evidence is well-established, and RCTs remain the ‘gold standard’ for regulatory and health technology assessment (HTA) submissions. However, adaptive pathways will likely necessitate the future use of non-RCT evidence in the HTA-decision-making process. To inform future submissions, we assessed acceptance based on non-RCT evidence across four HTA agencies, and explored agencies’ reactions to such study designs. METHODS: All single HTA appraisals in 2015 from NICE, CADTH, PBAC, and IQWiG were included in the analysis, including resubmissions. Multiple technology appraisals, vaccines, and requests for advice were excluded. The recommendation, reasoning behind the recommendation, and whether or not a non-RCT design trial (e.g. single-arm or observational) was included in the submission, were extracted.  RESULTS: A total of 189 appraisals were extracted: 49 for CADTH, 59 for PBAC, 36 for NICE, and 45 for IQWiG. Non-RCT evidence was considered in 5 CADTH appraisals (10%), 12 PBAC appraisals (20%), 13 NICE appraisals (36%), and no IQWiG appraisals. Only three treatments were given a positive recommendation solely on the basis on non-RCT evidence in 2015: daclatasvir for HCV GT3 infection (CADTH, PBAC), ponatinib for chronic myeloid leukemia (PBAC), and darunavir/cobicistat for HIV infection (CADTH). The remaining submissions included additional RCT evidence or were rejected due to high uncertainty about treatment effect. Non-RCT evidence did, however, contribute to a favorable outcome in some submissions, e.g. idelalisib for CLL and tolvaptan for PKD (NICE). CONCLUSIONS: NICE, CADTH, and PBAC will consider non-RCT studies such as real-world data and single-arm trials in certain circumstances, but are critical of the lower certainty of the evidence. The contribution of non-RCT evidence to the HTA-decision-making process is currently minor; however, as adaptive pathways and personalized treatment strategies gain momentum, HTA bodies will need to become more flexible in their approach to assessing non-RCT evidence.