Improving Transparency and Credibility of Real-World Evidence Research: The HARPER Protocol Template

Download Interpretive Report

WHAT IS THE HARPER PROTOCOL TEMPLATE ABOUT?

Healthcare decision makers—including government regulators, health technology assessment bodies, clinicians, and payers—are increasingly relying on studies that use data collected during routine clinical care (referred to as real-world data) to help guide decisions about which drugs, devices, and vaccines work safely and effectively in everyday practice.

This growing reliance on real-world evidence (ie, the insights and conclusions drawn from analyzing routine clinical data) is important because this evidence can offer timely, practical information about how treatments perform outside of strictly controlled research environments. Real-world evidence can also address questions that traditional clinical trials alone cannot always answer.

Despite the value of real-world evidence, a persistent problem has undermined confidence in many of these studies. Unclear, incomplete, or inconsistent reporting of how studies were designed, carried out, and analyzed makes it very difficult for independent reviewers to verify, trust, or reproduce the findings. High-profile cases in which studies were later retracted due to flawed methods or poor reporting on unsuitable data have intensified calls from researchers and decision makers for greater transparency.

A critical knowledge gap in the field has been the absence of a single, widely accepted international standard that research teams, regulatory agencies, and other decision makers can all use to ensure that study plans are properly documented before research begins. To address this, 2 major international organizations, the International Society for Pharmacoepidemiology (ISPE) and ISPOR—The Professional Society for Health Economics and Outcomes Research, joined forces to develop a common template for writing study protocols, which are the formal plans that describe how a study will be designed and carried out.

The resulting product, the HARmonized Protocol Template to Enhance Reproducibility (HARPER), represents a significant and concrete step forward in establishing shared expectations for how researchers should document and communicate the scientific decisions underlying their real-world evidence studies.

HOW WAS THE HARPER PROTOCOL TEMPLATE DEVELOPED?

The core principle guiding development of this report was that improving the clarity, completeness, and structure of study protocols could reduce ambiguity, prevent avoidable errors, and allow independent reviewers to more reliably evaluate whether a study is well-designed and whether its results can be trusted.

A task force was established to put this principle into practice, bringing together experts from the two professional societies and representatives from regulatory agencies, health technology assessment organizations, biopharmaceutical and device industry groups, and academic research institutions from multiple countries across Europe, North America, Asia, and Australia.

The task force met over several months to systematically identify, compare, and integrate existing protocol templates, develop a harmonized draft version, and then rigorously test that draft against both completed and in-progress real-world evidence studies.

The task force undertook a structured, expert-driven review and development process. Their approach included formal searches of scientific databases and professional networks to identify relevant existing guidance documents, systematic comparison of the structure and content of those documents, and iterative drafting of the new template with input from the full international group.

The primary focus of study was not on human patients, but rather a thorough review of existing international guidance documents, protocol templates, and real-world evidence studies to verify that it was both comprehensive and practical to use in real research settings.

This method was deliberately chosen because building on standards already developed and endorsed by major international organizations provided the strongest possible scientific foundation for the new harmonized template, while also maximizing the likelihood of broad uptake.

WHAT DOES THE HARPER PROTOCOL TEMPLATE CONTAIN?

A particularly striking finding during the development of the new harmonized template was that even among the 4 most widely used existing international protocol templates, there was substantial variation in the depth of guidance offered, the level of operational detail required, and the way study elements were sequenced and organized. This disparity helps explain why reproducing real-world evidence studies has proven to be so difficult and why a harmonized standard was so urgently needed.

The outcome of the task force’s collaborative effort was HARPER itself— a comprehensive, structured protocol template. HARPER features free-text narrative sections (where researchers describe the context and reasoning behind their methodological choices), organized data tables (where precise operational details of study variables, timelines, and analyses are specified), and a visual study design diagram (which gives reviewers a clear at-a-glance summary of the overall study structure).

The template addresses all elements considered essential for a complete real-world evidence study protocol:

• Research question
• Study population and how it is defined
• Treatments being compared
• Health outcomes of interest
• Data sources being used
• Statistical analysis plan
• Management of missing data
• Dedicated tracking table to document any changes made to the protocol as the study progresses

WHAT ARE THE STRENGTHS AND LIMITATIONS OF THE PROTOCOL TEMPLATE?

A notable strength of this work is that HARPER was developed through a rigorous, internationally inclusive process, drawing on contributions from stakeholders from multiple countries, and was then stress-tested against 5 real-world study examples to demonstrate that the template could be practically applied across a range of different research designs and data settings.

One meaningful limitation is that the structured tables within HARPER may present difficulties for studies using highly complex or newly emerging research designs that do not map neatly onto the template’s predefined fields, and the ordering of sections may not align seamlessly with the existing workflows or reporting conventions of every organization or regulatory system.

WHY IS THE HARPER PROTOCOL TEMPLATE IMPORTANT?

The template is of notable importance because when regulatory agencies, health technology assessment bodies, and payers cannot clearly understand or independently verify a study’s methods, they cannot confidently use its results to make sound decisions about approving, recommending, or covering treatments for patients. The HARPER protocol template helps address this challenge.

Many global organizations have already endorsed and adopted the HARPER protocol template. This could meaningfully change how studies are planned and described. It encourages research teams to think carefully about key design choices before collecting data, and it gives reviewers a shared format they can easily navigate and evaluate.

Patients stand to benefit indirectly because better-planned and more transparently reported studies produce more reliable evidence about how treatments actually perform in real-world clinical settings—not just under the highly controlled conditions of formal randomized clinical trials—meaning that patients’ coverage and prescribing decisions can be better grounded in evidence.

Over the longer term, consistent adoption of a harmonized protocol template like HARPER has the potential to reduce the number of poorly designed or inadequately reported studies entering the healthcare evidence base, thereby raising the overall quality, reliability, and usefulness of real-world evidence as a foundation for healthcare policy and clinical decision making globally.