OBJECTIVES: To gain insights into the degree of difficulty researchers have identifying, evaluating, and analyzing secondary real-world data (RWD) sources, such as claims and electronic health records, to generate RWE.

METHODS: An anonymous online survey was fielded (February-May 2024) that asked researchers to rate on a 7-point Likert Scale (1 most difficult, 7 least) the following issues: identifying & evaluating fit-for-purpose data sources; specifying a rigorous study design; identifying appropriate codes and developing algorithms to select study patients, classify them into treatment groups, assess covariates of interest, and specify measures of treatment outcomes; and selecting statistical methods for data analysis. Respondents also were asked to rank-order these items from most to least difficult.

RESULTS: A total of 53 researchers completed the survey. Most survey respondents worked for HEOR consultancies (34.0%) or life sciences companies (32.1%) and the vast majority had performed at least six RWE studies in the past five years (distribution: 1-5 studies, 15.1%; 6-19, 45.3%; 20-49, 26.4%; 50+, 13.2%). Respondents ranked evaluating RWD sources as #1 most difficult (median Likert rating of 3.0), followed by identifying fit-for-purpose RWD sources (3.0), specifying study outcomes (4.0), assigning patients to treatment groups (4.0), identifying study patients (4.0), specifying the study design (4.5), assessing patient covariates (5.0), and selecting statistical methods (5.0). Subgroup analyses reassuringly revealed that the more RWE studies researchers have performed the less difficulty they have across all facets.

CONCLUSIONS: Researchers find identifying and evaluating secondary data sources to be the most vexing aspects of using RWD to generate RWE. Developing code-based algorithms to create the analytic data files are somewhat less difficult, with selection of study design and statistical methods relatively straightforward in comparison. These findings underscore the continuing need for tools and guidance to alleviate difficulties in using secondary data sources to generate RWE, particularly for inexperienced researchers.