OBJECTIVES: The limitations of clinical trials have highlighted the importance of real-world evidence (RWE) for drug approval and reimbursement, especially for rare diseases. The FDA recognizes real-world data (RWD) including observational studies as key source of RWE and its RWE program will evaluate them for regulatory decisions based on study parameters. NICE also has evaluated technologies using observational studies for efficacy and cost data. However, Measurement errors and biased quality assessment (QA) compromise the accuracy and precision of RWD, that hinders robust evidence generation. Since diverse QA tools are used globally for comparative observational studies without standardization, this review aims to assess the tools used for cohort and case-control designs and highlight their strengths and limitations.

METHODS: A systematic search on MEDLINE and EMBASE identified systematic literature reviews (SLRs) published between 2019-2024 in which cohort and case-control studies were included. These SLRs were screened to identify the tools used for QA of cohort and case-control studies. The methodological aspects of the relevant tools were assessed.

RESULTS: Through screening, 340 studies were identified. The studies employed 22 different tools for QA of cohort and case-control studies. The National Institute of Health Quality Assessment Tool was utilized most often (41%), followed by the Joanna Briggs Institute checklist (18%), and the Newcastle-Ottawa scale (13%). The identified tools were either completely qualitative in nature or had an item-based equal weighted scoring. None of the tools were graded based on the different weight of items.

CONCLUSIONS: We identified that most of the quality appraisal tools for cohort and case-control studies were subjective in nature and the assessments were neither reproducible nor reflect true quality, leading to potential biases. Therefore, there is a need for a quantitatively graded checklist with appropriate weight distribution across components/domains for empirical quality assessment of cohort and case-control studies.