TOWARD OPEN SCIENCE FOR LARGE HEALTH CARE DATABASE RESEARCH- IMPROVING TRANSPARENCY AND REPRODUCIBILITY OF EVIDENCE
Author(s)
Elliott M. Antman, MD, Harvard Medical School, Boston, USA; Jessica Daw, PharmD, MBA, UPMC Health Plan, Pittsburgh, USA; Shirley V. Wang, PhD, Brigham and Women's Hospital and Harvard Medical School, Boston, USA; Richard Willke, PhD, ISPOR, Lawrenceville, USA
Presentation Documents
ISSUE: Decision makers, researchers and other key stakeholders recognize the need for greater transparency in methods and data behind research that informs decision making. Large health care database analytics has become a key source of evidence regarding safety and effectiveness of medical products in real-world settings. Unlike other biomedical research fields that rely on primary data collection, data acquisition is not the biggest roadblock for large healthcare database research. Health care databases can be licensed by many investigators, making it theoretically possible for independent investigators with licenses to the same databases to reproduce studies. However, the current lack of detail in public reporting of key operational and methodological decisions behind large healthcare database studies make it difficult to recreate analytic cohorts and findings from raw source data. Existing reporting guidelines lack sufficient detail on what is necessary to report for full transparency. Furthermore, the academic focus on novelty and innovation provides little incentive for research replication, and there are strong incentives to protect analytic intellectual property. It is unclear what steps can or should be taken to move the field of database analytics toward more open, transparent, and robust research practices. The panel of speakers will debate whether current initiatives to improve database research reproducibility and transparency can successfully prompt field-wide change in database research conduct and reporting.
OVERVIEW: Dr. Willke will frame the issues and outline ISPOR’s strategic initiatives for promoting and improving open science for research using real-world data. What are potential barriers? Dr. Antman will discuss challenges that national efforts to improve transparency and reproducibility of randomized controlled trials have encountered, focusing on barriers to executing change and what researchers generating real-world evidence from health care databases can learn from those efforts. Can decision makers create incentives to change research behavior? Dr. Shrank directs a payer organization that makes many decisions based on real-world evidence. He will discuss potential “carrot” and “stick” incentives that could raise the bar for research practice and reporting on database analytics. Are current efforts to improve database research reproducibility likely to be successful? Dr. Wang will discuss a multi-stakeholder task force to improve reproducibility and transparency of database research for decision makers, results from reproduction of 32 published database studies, as well as standards for research transparency in large distributed database networks such as FDA’s Sentinel and PCORnet. She will assess whether incentives built into current initiatives have enough teeth to prompt widespread change in research behavior. Dr. Willke will then lead a Q&A session with the panel and audience regarding the pros, cons, and likely effectiveness of various approaches to improving reproducibility. Are there more effective approaches that could align incentives and move the field toward greater research transparency, rigor and trust?
Conference/Value in Health Info
2017-05, ISPOR 2017, Boston, MA, USA
Code
IP19
Topic
Methodological & Statistical Research, Real World Data & Information Systems