Consistency of Published Observational/Real-World Studies With the STROBE Guidelines: An Analysis of Three Indications

OBJECTIVES: As acknowledged by the ISPOR/ISPE task force, improving transparency in reporting real-world data is essential. However, the compliance of published observational studies with reporting guidelines is unknown. This targeted literature review assessed the transparency of reported data from observational studies in three indications, evaluating published articles using the STROBE guidelines/checklist.

METHODS: Observational studies were identified in EMBASE using conventional search methodology based on ISPOR recommendations in three indications within the fields of dermatology/oncology/genetic disease. Years 2017/2020/2023 were selected to explore longitudinal changes. Open-access publication (itself a transparency goal) in a ‘top25%’ journal (according to Journal Citation Reports^TM) was required for inclusion. Articles were scored 1(yes), 0(no), or not applicable for the presence of each of the 34 STROBE checklist items/sub-items. Scores (range 0–1) were calculated per article section and per article as the proportions of applicable checklist items with 'yes'. Nonparametric Kruskal-Wallis tests compared scores between years and indications.

RESULTS: Among 132 included articles (dermatology 62; oncology 21; genetic disease 49; 2017: 32; 2020: 34; 2023: 66), the mean[SD] overall score was 0.73[0.13]. The mean[SD] overall score was not significantly different across years (2017: 0.74[0.11]; 2020: 0.73[0.14]; 2023: 0.72[0.14]; P=0.66) or therapeutic area (dermatology 0.71[SD]; oncology 0.76[SD]; genetic disease 0.74[SD]; P=0.12). Overall, mean[SD] section scores were title/abstract 0.81[SD]; introduction 0.93[SD]; methods 0.70[SD]; results 0.72[SD]; discussion 0.87[SD]; other information 0.75[SD]. The most frequently reported items/sub-items were ‘title/abstract: informed and balanced’ (126/132[95.5%]), ‘descriptive data: characteristics’ (127/132[96.2%]), ‘key results summary’ (126/132[95.5%]), and ‘interpretation’ (127/132[96.2%]). The least frequently reported items/sub-items included ‘participants flow diagram’ (27/126[21.4%]), ‘bias’ (methods) (30/132[22.7%]), and ‘statistical methods: sensitivity analyses’ (29/98[29.6%]).

CONCLUSIONS: Our review observed frequent transparency deficiencies in methods and results sections of peer-reviewed observational studies across selected indications, with no significant differences overtime or between indications. These results highlight the need for improved adherence to transparency guidelines in real-world data reporting.