Co-Chairs:
	Marc Berger MD Vice President, Global Health Outcomes Eli Lilly and Company Indianapolis, IN, USA			Sharon-Lise Normand PhD Professor of Health Care Policy (Biostatistics), Harvard Medical School and Professor, Harvard School of Public Health Boston, MA, USA
Leadership Team:
	Fred Anderson PhD Research Professor of Surgery Director, Center for Outcomes Research University of Massachusetts Medical School Worcester, MA USA			Thomas Ten Have PhD, MPH Professor, Center for Clinical Epidemiology and Biostatistics University of Pennsylvania School of Medicine Philadelphia, PA USA
	Nancy Dreyer PhD, MPH Chief of Scientific Affairs & Sr. Vice President, Outcome Cambridge, MA, USA			Adrian Towse MA Director, Office of Health Economics London, England, United Kingdom
	Art Sedrakayan MD, PhD Center for Devices and Radiological Health FDA Silver Spring, MD, USA

Goal:
The goal of this Task Force is to provide good research practices for prospective observational clinical studies that focus on the effectiveness and/or comparative effectiveness of health care interventions.

Background / Overview:
Well-designed observational research has an important and valuable role in generating data about comparative effectiveness in the context of real clinical practice settings. Recently, the ISPOR Retrospective Database Analysis Good Research Practices Task Force completed its work with a focus on comparative effectiveness research [See: Defining, Reporting and Interpreting, Approaches to Mitigate Confounding and Bias, and Analytic Methods to Improve Causal Inference.] While many of the issues addressed by this Task Force also may apply to the design and conduct of prospective observational studies, additional issues must be addressed when designing a prospective observational study that is “fit-for-purpose” – that is, it is designed to prospectively test hypotheses about comparative effectiveness in patients recruited to participate in an observational clinical study.  This Prospective Observational Clinical Study Good Research Practices Task Force will focus on good research practices for prospective observational clinical studies, not patient registries. Patient registries may accept all patients with an exposure (e.g. lead) or with specific circumstances (e.g. pregnancy) and for which effectiveness hypotheses are not pre-specified - registries being more commonly used for pharmacovigilance or descriptive purposes. However, it is recognized that when registry data are accessed to examine hypotheses regarding comparative effectiveness, they are indistinguishable from prospective observational studies.

To enhance the credibility and reputation of prospective observational clinical studies for comparative effectiveness, researchers from academia, industry, and research organizations must be exemplary in execution and transparent in communication when performing and reporting scientific investigations. They should adhere to the highest standards in conducting meaningful research. Individual researchers face a wide range of choices in how to do such studies.  Such studies should be carried out in compliance with generally accepted good practices of outcomes research and pharmacoepidemiology. This Task Force Report will provide expert guidance on how to weigh various design options and how to evaluate the quality of prospective observational clinical studies which should be useful to the research community, policy makers, payers, and the public..

Previous work in the development of good research practices for observational clinical studies are: a) the European Union Medicines Agency’s pharmacoepidemiology and pharmacovigilance project (ENCePP) in 2010, which is a code of conduct to help guarantee high standards, scientific independence, and transparency in post-authorization studies - the ENCePP also created a checklist of methodological research standards for ENCePP studies; b) the US Agency for Healthcare Research and Quality’s (AHRQ) “User’s Guide to Registries Evaluating Patient Outcomes” describing the appropriate use of patient registries published in 2007, directed primarily at specific types of observational designs; c) the International Society for Pharmacoepidemiology (ISPE) Guidelines for Good Pharmacoepidemiology Practices (1996) and updated (2008). In the United States, the passage of health care reform is placing new resources into the development of additional comparative effectiveness information and research.  Such research often incorporates a wide variety of observational study designs to enhance randomized clinical efficacy studies. AHRQ has conducted conferences on methodologic issues to be addressed in observational research (Medical Care, October 2007) and reports from a subsequent conference are forthcoming (June 2010).
 
Thus, although guidelines and recommendations for observational studies exist to one degree or another, this Task Force is specifically addressing prospective observational clinical studies. Some of the analytic challenges inherent in prospective observational studies could be addressed by employing a randomized naturalistic effectiveness study design – what has been called a “pragmatic” or “practical” study or large simple trial.  This Task Force will also consider the extent to which such a study may or may not address various analytic issues including confounding and bias.

The following are the topics to be addressed by the members of this Task Force:

1. When is the design and conduct of an observational study adequate to provide probabilistic inference about causality?  More specifically, when does a study provide information robust enough to inform policy decisions regarding comparative effectiveness? 
   Formal sensitivity analysis to assess unmeasured confounding; design adjustments such as multiple control groups to assess confounding and assay sensitivity
   
   
2. When is the design and conduct of an observational study “fit for purpose” to answer questions of comparative effectiveness?
   Ideally, prospective observational clinical studies should recruit a population of patients that is representative of what occurs in real-world practice and will be sufficient to meet the aims of the study.  The use of explicit, a priori inclusion/exclusion criteria should make transparent the comparability (or differences) between patients enrolled in the study and those treated in everyday clinical practice  For many research questions, enrollment should comprise a heterogeneous population with new and existing patients.
   
   Prospective parallel cohort studies are commonly employed to examine comparative effectiveness of alternative therapeutic strategies; with respect to these study designs, multiple-arm studies with broad inclusion are considered optimal, but under what circumstances (either methodological or practical) are other design choices appropriate? Can single arm prospective clinical studies of effectiveness provide useful information and how would one ensure that it was truly observational in nature (i.e. that the conduct of the study did not provide incentive for treatment assignment)?  A single-arm study where tens of thousands of patients are all prospectively receive a sponsor’s product may rightly be perceived as a “seeding trial” rather than a scientific endeavor.  While one can readily calculate an appropriate sample size for a comparative study, enrollment may not proceed evenly among different study groups (i.e. accrual of patients may occur more rapidly in one arm than another).  How should this be addressed?  Can one or more study arms in a prospective observational clinical study be overpowered?  What options are there to address this?  Capping schemes may introduce analytic problems and may impact the natural behavior of practitioners, impacting treatment assignment.  What if one arm is very slow to enroll?  Should there be interim futility analyses built into the protocol to address the question of whether the study can be completed in a relevant timeframe or for the available budge?  What standard or expectation should there be about the proportion of eligible study participants who actually agree to participate and provide data?  What about expectations regarding patient withdrawals? How should this standard differ from that applied to RCTs (if at all), and why?
   
   The opportunity for confounding and bias in observational studies is a critical issue that must be addressed in the analytic plan.  The ISPOR Retrospective Database Studies Good Research Practice Task Force examined many of these issues.  The report of this ISPOR Good Research Practices Task Force will be revisited to examine what recommendations would be unchanged, and what might differ for the analysis of prospective observational clinical studies.  The ability for confounding and bias to be addressed by randomly assigning treatment in a pragmatic study design will also be explored. What is the intervention you are comparing (drug vs. process of care) such as issues of switching or discontinuation will also be examined.
   
   
3. Are there different issues involved when one is examining the comparative effectiveness of biologics, drugs, devices, procedures, or programs of care?
   
   These questions will be answered in the context of the following:
   
   
   • Noninterventional nature of prospective observational clinical studies.  Observational research is characterized by non-intervention in assignment of treatment.  This contrasts with pragmatic studies where treatment is randomly assigned. Many factors may influence the assignment of an intervention including evidence-based guidelines, provider formularies, health practitioner training, health practitioner recommendations, patient preferences and out-of-pocket costs.  The contribution of these factors to patient outcomes and comparative effectiveness analyses needs to be addressed by a robust analytic plan.  However, it is integral to observational studies that the assignment of treatment not be dictated by the protocol. While a protocol may require additional data collection beyond what would be routinely performed, it does not interfere with the normal decision-making process involved in therapeutic choice or subsequent clinical management. Protocols may limit enrollment to defined patient groups to meet study objectives according to its analysis plan. Given the broadness with which the term “observational study” is commonly applied (even with the qualifier of “prospective”), a clear taxonomy of study types and their distinguishing features (including strengths and limitations with regard to threats to validity) would in itself be a useful reference.
     
     
   • Safety reporting. For phase IV studies, investigators need to consider specific requirements to collect safety information. Following the guidelines for randomized clinical studies may not always be necessary or appropriate, and may confound the real-world behavior of practitioners and patients. Eliciting adverse events may itself be considered an intervention. Should safety issues be allowed to emerge in a naturalistic fashion and dealt with as a real-world circumstance? [8]
     
     
   • Reporting and publication. Publication bias is a major factor to consider in observational research; hence, investigators may need to disclose analysis plans early and publish protocols creating visibility and transparency.  The prior retrospective database taskforce made recommendations regarding reporting and publication.  These will be revisited and recommendations regarding how they might apply or be adapted to prospective observational clinical studies will be made.
     
     
   • Appropriate remuneration for researchers and for subjects/participants.  As for any clinical study, investigators should be paid only for study-specific time and expenses incurred to conduct the research. What are, if any, appropriate incentives for patient participation in a study? Are there reasonable levels of remuneration for participants that do not skew their assessment of risk in deciding to participate? Also “types” of remuneration can be important (cash vs. gift card vs. something “material” vs. you name it). 

REFERENCES

1. European Medicines Agency. The ENCePP Code of Conduct for Independence and Transparency in the conduct of Pharmacoepidemiological & Pharmacovigilance. Consultation Draft 13 November 2009.
2. European Medicines Agency. (2009). Checklist of Methodological Research Standards for ENCePP Studies: Draft for public consultation (No. EMEA/540136/2009). London, UK.
3. Registries for Evaluating Patient Outcomes: A User’s Guide (No. AHRQ Publication No. 07-EHC001-1). (2007).  Rockville, MD: Agency for Healthcare Research and Quality.
4. International Society for Pharmacoepidemiology. ISPE guidelines for good pharmacoepidemiology practices (GPP). Pharmacoepidemiology and Drug Safety 2008; 17: 200–208.
5. von Elm E, Altman D G, Egger M et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies. Ann Intern Med 2007;147:573-577.
6. European Union. Directive 2001/20/EC on the approximation of the laws, regulations and administrative provisions of the Member States relating to the implementation of good clinical practice in the conduct of clinical studies on medicinal products for human use. Official Journal of the European Communities. 2001
7. Medicines Control Agency, Committee on Safety of Medicines, Royal College of General Practitioners, British Medical Association and Association of The British Pharmaceutical Industry. Guidelines for company-sponsored Safety Assessment of Marketed Medicines (SAMM guidelines) , Br J clin Pharmac 1994; 38: 95-97
8. Committee for Medicinal Products for Human Use. The Rules Governing Medicinal Products in the European Union – Guidelines on Pharmacovigilance for Medicinal Products for Human Use. 2008 Vol 9a
9. European Federation of Pharmaceutical Industries and Associations. EFPIA code on the promotion of prescription-only medicines to, and interactions with healthcare professionals 2007

If you are interested in joining this ISPOR Task Force, please complete the Special Interest Group and Task Forces Form.

Task Forces Index