OBJECTIVES

: Retrospective data analysis (RDA) is the most common design of real world evidence (RWE) studies. Available recommendations for RDA are focused on methodology or generalized for all designs of observational studies. We customized a consolidated checklist to standardize the reporting of RDA.

METHODS

: Current guidelines for RWE (ISPOR reports, STROBE, RECORD, PCORI methodology, ENCePP, GRACE) were reviewed. Consensus checklist items (e.g, clear objective, eligibility criteria, covariate assessment, statistical methods, source and magnitude of potential bias, limitations) were generated and assigned weighted scores based on item significance. In a pilot study, the checklist (StRWE-Retro) was used by 4 reviewers to assess the quality of selected RDAs independently. Studies achieving ≥75% of total possible score were considered high checklist score (HCS). The effect of journal impact factor (IF) (<5 and ≥5) was analyzed. Internal reliability (consistency among reviewers) of the checklist was assessed using the Cronbach Alpha model (two-way mixed) at 95% CI in SPSS (Version-23).

RESULTS

: StRWE-Retro has 28 items, each item score ranging 1-9 with a total score of 170. The mean inter-item correlation was 0.77 (range 0.53, 1.0) with an interclass correlation coefficient 0.93 (95% CI: 0.87, 0.97). The Cronbach Alpha for inter-item reliability was 0.91. Of 76 studies assessed, 40 (52.6%) were HCS. A total of 36 (47.4%) studies were published in high IF journals and 40 (52.6%) in low IF journals. Journal IF did not appear to affect the frequency of HCS vs non-HCS (p=0.16). The two least reported items were bias (n=20, 26.3%) and patient disposition (n=21, 27.6%).

CONCLUSIONS

: The StRWE-Retro checklist showed good internal consistency. The results suggest that reporting quality of RDA is not significantly related to journal IF. The StRWE-Retro checklist may improve adherence to recognized reporting standards for RDA; further validation by a larger panel of reviewers is warranted.