Data Reliability in Retrospective Chart Review Studies: Results and Considerations From a Novel Data Review Methodology
Author(s)
Neil R. Brett, PhD1, Luke Safarcyk, BA2, Elizabeth Donahue, BSc3, Marielle Bassel, BA1.
1PPD™ Observational Studies, Thermo Fisher Scientific, Montreal, QC, Canada, 2PPD™ Observational Studies, Thermo Fisher Scientific, Covington, KY, USA, 3PPD™ Observational Studies, Thermo Fisher Scientific, Boston, MA, USA.
1PPD™ Observational Studies, Thermo Fisher Scientific, Montreal, QC, Canada, 2PPD™ Observational Studies, Thermo Fisher Scientific, Covington, KY, USA, 3PPD™ Observational Studies, Thermo Fisher Scientific, Boston, MA, USA.
OBJECTIVES: Retrospective chart review studies using remote data review often struggle with data reliability issues. Chart reviews requiring source data verification (SDV) may face challenges in implementation, site interest, data protection and timelines. We aimed to describe an alternative method for data quality review and highlight key considerations.
METHODS: Three global chart review studies using repeat abstraction of key variables data review method are described. For this method, key variables were placed into paper case report forms (CRFs), sites re-abstracted variables for a pre-defined number of patients, and re-abstraction CRFs were compared to the original CRFs for discrepancies. Learnings and considerations are provided.
RESULTS: The three studies were in gastroenterology, rare disease and immunology, across six countries and enrolled a total of 275 patients. For each study, 1-7 pages of key variables were selected prior to data abstraction initiation, for sites to re-abstract. Sites performed re-abstraction for 1-3 patients (study-dependent; 42 total patients). Across studies, there were no abstraction discrepancies for 21% of patients, <10% discrepancies for 2%, 10-39% discrepancies for 67%, and >40% discrepancies for 10% of patients. Sites corrected discrepancies, and study-specific quality control (QC) plans specified any other actions sites needed to take (e.g., re-training, etc). This method suits non-regulatory or regulatory studies looking for additional QC (when SDV is not feasible), and/or with complex data abstraction (i.e., unstructured notes). A study-specific QC plan should specify variables for re-abstraction, numbers of patients, and handling discrepancies. Different staff should perform original and re-abstraction to test inter-rater reliability. Study timelines and site burden should be considered when writing the QC plan.
CONCLUSIONS: Repeat abstraction of key variables data review methodology demonstrated the importance of additional data QC in these chart review studies. This methodology is cost efficient and should be tailored to each study based on study design/outcomes, timelines and other factors.
METHODS: Three global chart review studies using repeat abstraction of key variables data review method are described. For this method, key variables were placed into paper case report forms (CRFs), sites re-abstracted variables for a pre-defined number of patients, and re-abstraction CRFs were compared to the original CRFs for discrepancies. Learnings and considerations are provided.
RESULTS: The three studies were in gastroenterology, rare disease and immunology, across six countries and enrolled a total of 275 patients. For each study, 1-7 pages of key variables were selected prior to data abstraction initiation, for sites to re-abstract. Sites performed re-abstraction for 1-3 patients (study-dependent; 42 total patients). Across studies, there were no abstraction discrepancies for 21% of patients, <10% discrepancies for 2%, 10-39% discrepancies for 67%, and >40% discrepancies for 10% of patients. Sites corrected discrepancies, and study-specific quality control (QC) plans specified any other actions sites needed to take (e.g., re-training, etc). This method suits non-regulatory or regulatory studies looking for additional QC (when SDV is not feasible), and/or with complex data abstraction (i.e., unstructured notes). A study-specific QC plan should specify variables for re-abstraction, numbers of patients, and handling discrepancies. Different staff should perform original and re-abstraction to test inter-rater reliability. Study timelines and site burden should be considered when writing the QC plan.
CONCLUSIONS: Repeat abstraction of key variables data review methodology demonstrated the importance of additional data QC in these chart review studies. This methodology is cost efficient and should be tailored to each study based on study design/outcomes, timelines and other factors.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
SA27
Topic
Study Approaches
Disease
Gastrointestinal Disorders, Infectious Disease (non-vaccine), Rare & Orphan Diseases