GRP logo
Retrospective Database Analysis – Bias & Confounding in the Design

Good Research Practices for Comparative Effectiveness Research: Approaches To Mitigate Bias And Confounding In The Design Of Non-randomized Studies of Treatment Effects Using Secondary Databases – Report of the ISPOR Retrospective Database Analysis Task Force – Part II

The citation for this report is:
Cox E, Martin BC, Van Staa T, Garbe E, Siebert U, Johnson ML, Good research practices for comparative effectiveness research: approaches to mitigate bias and confounding in the design of non-randomized studies of treatment effects using secondary data sources: The ISPOR good research practices for retrospective database analysis task force–Part II. Value Health 2009;12:1053-61.

For Part 1 of the ISPOR Retrospective Database Analysis Task Force Report:
Good research practices for comparative effectiveness research: defining, reporting and interpreting nonrandomized studies of treatment effects using secondary data sources: The ISPOR good research practices for retrospective database analysis task force report—Part I.
http://www.ispor.org/TaskForces/documents/RDPartI.pdf

For Part III of the ISPOR Retrospective Database Analysis Task Force Report:
Good research practices for comparative effectiveness research: analytic methods to improve causal inference from nonrandomized studies of treatment effects using secondary data sources: The ISPOR good research practices for retrospective database analysis task force report—Part III. http://www.ispor.org/TaskForces/RetrospectiveDBPractices3.asp

Task Force Chair:
Michael Johnson PhD, Associate Professor University of Houston, College of Pharmacy, Houston, TX, USA

Task Force Members:
David Atkins, MD, Assoc Director, HSR & D, Dept. of Veterans Affairs Health Services, Washington, DC , USA
Marc Berger MD,
Vice President, Global Health Outcome, Eli Lilly and Company, Indianapolis, IN, USA
Emily Cox, PhD
, Sr. Director of Research, Express Scripts, St. Louis, MO, USA
William Crown PhD,
President, i3 Innovus, Waltham, MA, USA
Colin Dormuth, ScD, Assistant Professor, Dept. of Anesthesiology, Pharmacology & Therapeutics, University of British Columbia; Chair of Pharmacoepidemiology Group, Therapeutics Initiative, Vancouver, BC, Canada
Edeltraut Garbe, MD, PhD
, Head of the Department of Clinical Epidemiology, Bremen Institute for Prevention Research and Social Medicine, Bremen, Germany
Muhammad Mamdani, PharmD, MA, MPH
, Director of the Applied Health Research Centre of the Li Ka Shing Knowledge Institute of St. Michael's Hospital and Associate Professor at the University of Toronto,
Toronto, Ontario, Canada
Bradley Martin PhD, RPh,
PharmD Assoc. Prof. and Division Chair, College of Pharmacy, University of Arkansas for Medical Sciences, Department of Pharmacy Practice, Little Rock, AR, USA
Uwe Siebert MD, MPH, MSc, ScD
, Professor of Public Health, University of Health Sciences, Hall, Austria
Tjeerd Van Staa PhD, MD, MSc, MA
, Head of Research, GPRD, London, UK

Background
With the advent of the Medicare Prescription Drug, Improvement and Modernization Act of 2003 (Medicare Modernization Act or MMA), extending pharmaceutical coverage to 42 million Medicare beneficiaries, the need for more information on the safety, effectiveness, and cost-effectiveness of drugs in everyday use has never been more critically needed. Indeed, the Medicare Modernization Act included in Section 1013 states a specific statement of need for more comparative clinical effectiveness studies.

Observational data provide a wealth of information on drug treatment in everyday practice, where long-term safety and effectiveness of drugs could be estimated. The major difficulty to overcome with estimation of true treatment effects (causal effects) from observational data is the presence of confounding factors affecting both treatment and outcome. Standard statistical approaches such as logistic regression for binary outcomes and proportional hazards regression for time-to-event outcomes are familiar and adequate methods to adjust for traditional confounding. With the proliferation of longitudinal data, with time-varying measurement of exposure and disease outcome, these methods have been shown to be biased in the presence of time-varying confounding. Methods such as marginal structural models using inverse probability of treatment weighting have been developed, but are not as well understood or in wide application as now more traditional methods such as propensity scoring.

Observational studies using retrospective data from large administrative and clinical ‘claims type’ databases contain a wealth of information which could be used to supplement the findings from randomized trials on the safety and effectiveness of drugs in routine clinical practice. The key feature of retrospective database studies that limits their usefulness and adoption of findings from these studies into policy and practice is that the observational design and resulting statistical control of confounding provides a weaker framework for internal validity and especially causal inference of exposure-disease associations than experimental designs. A full exploration of good research practices for retrospective databases will include an over-arching view toward methods that address this concern and that attempt to guard against threats to internal validity and improve causal inference.

Draft Final Report
The Task Force presented their final draft report including responses to member comments at ISPOR 14th Annual International Meeting in Orlando, Florida, May 2009.  This session focused on creating a guidance document on state of the art approaches for comparative therapeutic effectiveness studies using secondary databases, with an overarching view toward design, analysis, and reporting of findings to inform policy and practice. The task force’s presentation entitled “DESIGN AND ANALYSIS OF NON-RANDOMIZED STUDIES OF TREATMENT EFFECTS USING SECONDARY DATABASES” can be viewed at http://www.ispor.org/TaskForces/documents/14thmeeting_retroforum.pdf


ISPOR Good Practices For Outcomes Research Index