The Official News & Technical Journal Of The International Society For Pharmacoeconomics And Outcomes Research

Real-Life Data: A Growing Need

Lieven Annemans PhD, MSc, Principal HEOR, IMS Health, Miesse, Belgium, Michael Aristides MSc, BA, Principal HEOR, IMS Health, London, UK, and Maria Kubin MD, MSc, Director, Global Health Economics and Reimbursement, Bayer Healthcare AG, Wuppertal, Germany

It is increasingly recognized that conclusions drawn from classical clinical trials are not always a useful aid for decision-making - assessing the value of a drug or technology requires an understanding of its impact on current management in a practical, real-life setting. But as the benefits of real-world data become more apparent so, too, do issues around its appropriate collection and reliability. Lieven Annemans, Principal HEOR, IMS Health, Michael Aristides, Principal HEOR, IMS Health, and Maria Kubin, Director, Global Health Economics and Reimbursement, Bayer Healthcare AG, consider some of the issues.

Analyzing real-life data for health economic evaluations is still a relatively young discipline as witnessed by the scarcity of good quality publications on real world studies in the medical literature. To a certain extent this reflects the difficulties involved in moving from the controlled and experimental environment of a clinical trial to a real world situation. Just how experimental we choose to be in our real life studies can be part of the problem.

What is ‘Real Life’ Data?
There are many, very varied, definitions of what constitutes ‘real life’ data: an ISPOR task force dedicated to investigating the collection of real life data has described it as everything that goes beyond what is normally collected in the Phase III clinical trials program in terms of efficacy; according to the European Forum “Relative Effectiveness” Working group it is, “a measure in understanding health care data collected under real life practice circumstances”; at the payer level a real-life study has been defined as “anything that is not interventional”.

Driving the Push for 'Real Life' Data
Aside from the question of why there is a current move to collect real-life data, there is also the issue of why the move is happening so quickly and in particular receiving so much attention. Several factors are driving the trend:

  1. Randomized Controlled Trials (RCTs), although recognized as the ‘gold standard’ for establishing efficacy, operate in an idealized environment and can only measure efficacy in limited populations. As such, they cannot provide a true indication of effectiveness - an area on which more and more knowledge is being sought.
  2. Real-life data can be obtained from sources that cannot be included in an RCT, and provide additional insights into areas such as epidemiology, or cross-data from a more naturalistic environment. Compliance, adherence and cost insights can also be obtained.
  3. Perhaps the most important driver of the push is that decision makers are expressing a much bigger interest in real life data on effectiveness than they ever have done before - to better manage uncertainty at the time they are making reimbursement decisions.

According to a recent survey by the European Commission, an increasing number of countries revisit the relative efficacy and effectiveness of a new treatment in comparison to standard therapy as part of their decision making processes for reimbursement and pricing. This process takes place in addition to the EMEA assessment, and aims at evaluating the added value offered by a new treatment. With a number of countries now moving into reference pricing for new treatments without added medical value, this additional effectiveness assessment has a major impact on a new drug’s success in the market.

When health authorities were asked what data they use for effectiveness, most responded with an admission to basing it on RCTs, despite knowing that randomized clinical trial data does not represent effectiveness. Some institutions accept modeling approaches as a bridge from efficacy to effectiveness.

Sources of Real Life Data
Real life effectiveness data can be collected in a number of ways, most commonly from:

  1. Databases: These include cross-sectional and longitudinal databases which essentially provide retrospective data but increasingly offer the opportunity to have prospective add-ins, e.g. on quality of life.
  2. Patient and population surveys: Primarily for epidemiological information.
  3. Patient chart reviews: Used to reflect particular insights in patient management.
  4. Observational data from cohort studies: What most people would understand by real life studies.
  5. Pragmatic clinical trials: Whether these are strictly “real-life” studies is open to debate. In a way they are simple experimental trials, which raise questions regarding the extent to which they reflect what is happening in real life. Efforts are however made to mimic a real life situation as much as possible.
  6. Registries: The use of registries - both large and small - is increasing. These involve registering and subsequently analyzing all patients treated at a particular centre for a particular condition on a continuous basis.

The use and value of databases varies greatly according to the point in time at which the real-life data are collected. Pre-launch collection invariably focuses on epidemiological data and on data to gain an understanding of the way patients are currently managed and to identify unmet medical needs. It can also be used to determine the costs associated with a particular disease or particular events that may be more common with one treatment than with another and, in terms of longer term outcomes, with the current standard of care. This real-life data is then combined with data from clinical trials to serve as an input for cost-effectiveness and budget impact models.

An example of real-world data collected in the pre-launch phase would be the case of a clinical trial testing Treatment A as existing therapy and Treatment B as a new treatment, which revealed different degrees of failure. It is helpful to complement this with additional information that is relevant to the reimbursement authority [Figure 1].

The questions then arise: “What is the actual incidence of failure in real life?” “How does it relate to what we’ve seen in the randomized clinical trials?” “What is the burden of the disease to the patient?” This information is rarely collected in the phase III clinical trials that primarily aim at regulatory approval. Furthermore, the burden may differ in the wider patient population from the more restricted cohort that is assessed in the phase III programme. Data relating to the cost and management of failures in real life are also hard to collect in a clinical trial because of all the protocol-related design issues and cost.


Observational Data Collection
Following launch, opportunities open up for direct comparative real-life trials to be conducted. These are increasingly requested by reimbursement authorities to validate the cost-effectiveness and budget impact models. They offer the opportunity to recruit a more representative patient population and assess clinically relevant endpoints, rather than the often more shortterm surrogate endpoints. Hence, these real life studies are a way to collect effectiveness data of a new drug in comparison to other key treatments in the market. Real life data also provides longer-term knowledge about safety and costs - again, something often requested by authorities.

Issues with Real Life Data Collection
There are a number of issues which confound the collection of real-life data. In Europe, for example, there is a lack of good quality and sufficiently representative databases in many countries - very different to the United States. Those that exist are often not complete across different health care centers. They may be focused on GPs or the hospital sector, but rarely cover all the different settings that play a role in medical treatment. They are also often missing data or contain poorly specified information (e.g. on the severity of the condition).

A further limitation is that the description of an event often differs in real-life data compared to a randomized trial. Take for example, the finding of deep vein thrombosis based on venography in a clinical trial - when it may not have become symptomatic in real life. In reality, only symptomatic cases are compared and asymptomatic events are not recognized. Nevertheless, both can result in a pulmonary embolism, and hence play a role. It is however hard to judge the frequency and impact of asymptomatic events in real life. So it is easy to end up comparing apples and pears when looking back into those databases.

Prospective data brings its own challenges, particularly in terms of the manpower effort and budget required to collect it - finding sources that are willing to provide the data can be an issue. Even with a careful controlled design, there may be various sources of bias that can completely distort the results. One typical example here is the case that different degrees of severity of a condition are treated with different drugs. In consequence, a treatment that is actually more effective may look less successful in an observational study, if it is administered to the very severe cases only, while the competing drug is mainly provided to the mild patients. Thorough statistical methods can control parts of these biases, however the risk of misinterpreting studies without randomized drug allocation remains high. With uncontrolled data, there are even more sources of bias and confounding. Treatment patterns sometimes differ considerably from one country to the other so prospective research must often be conducted in a series of countries to enable a meaningful picture to be presented to the local reimbursement authorities.

Despite these issues, there is at least one good reason to work on observational data and real life data before launch. At the time of the reimbursement decision, payers are increasingly asking for more evidence of cost effectiveness that applies to the real world. Normally, this evidence is not available so the industry argues for the fact that collecting real-life data on the new drug is only possible after a decision regarding reimbursement. As a result, the majority of payers have started to accept and even request modeling. This modeling is based on the different pieces of available observational data in addition to evidence from RCTs. Some payers provide conditional approval pending the collection of real-world data collected post-launch for validation. However, this is not common practice across all countries. In those countries where modeling is not accepted or where the assumptions in the model are not accepted, reimbursement and pricing may then be negatively impacted. In consequence, it is very recommendable to have a solid value package with considerable real life evidence before launch.

Benefits of Post-launch Collection
The first key benefit of post-launch collection is the opportunity to validate modeling assumptions. There is also the real ability to prove effectiveness in a large pragmatic trial against key competitors which addresses additional outcomes like adherence, compliance parameters and sometimes more long-term clinical events. The data can be much more representative and adherence to guidelines can also be reflected, i.e., use in clinical practice compared to guideline recommendations. Finally, a range of treatment options can be presented and the changing environment evaluated.

Retrospective or Prospective?
The main issues surrounding retrospective research are:

  • Shortage of sources (countries) and databases (health care providers)
  • Lack of precision - in terms of diagnosis and outcomes
  • Incompleteness
  • Confounders and bias

Also in prospective research there are still a number of challenges including:

  • Identifying where care is provided: It is not easy to identify where a subject is really being treated for a particular condition and once you start designing where they should be treated, you already end up interfering with real life events.
  • Data quality: This varies between community-based settings versus the academic context - if GPs in a regular community do not have a particular interest in a specific condition they are likely to produce very different data points and degrees of completeness.
  • Design issues: Particularly around comparators and endpoints - if a patient treated successfully does not return, there is no control visit to confirm the effects at that stage so it is never really possible to determine how many of the patients who did not return were so dissatisfied that they opted to go to another GP or specialist.
  • Selection bias: If not randomized, selection bias cannot be completely circumvented.
  • Assessment bias: This can only be avoided if the assessment is blinded, because the patient or physician may act differently knowing a treatment is being used compared to another. However, once you blind and randomize an assessment, it moves towards an RCT assessment.
  • Hawthorne effect: This refers to the famous effect whereby patients and physicians in a clinical trial behave differently simply because of the fact that they are in a trial and being observed.


In its various forms - longitudinal or cross-sectional, retrospective or prospective - observational patient data has an important role to play in the evaluation of epidemiology and burden of disease, treatment patterns, compliance, persistence, and health outcomes of different treatments.

It can add value during all phases of Health Economics and Outcomes Research (HEOR) activities for a drug. In the early development stages, it supports a better understanding of disease epidemiology and medical practice. Later on it provides input into the development of value dossiers and cost effectiveness models and patient evaluation. Post-launch, through observational studies, it may help to validate previous database and modeling work as well as supporting decision makers requiring reviews or substantiation of claims and appropriate use. It is perhaps most valuable to parameterise and help extrapolate economic and budget impact models beyond the controlled environment of the experimental RCT, supporting the need to evaluate the impact of interventions before their introduction and in the absence of empirical data.

Internationally, real-life data is increasingly used in the development of clear, evidence-based documentation for demonstrating value. With the diverse and growing number of stakeholders making treatment and purchasing decisions today, demonstrating product value in both clinical and economic terms is critical to achieving successful reimbursement and central to enabling the costs and benefits of competing health technologies to be quantified for funding decisions. The hierarchy of evidence that has been in place for so long is thus beginning to change: people who have celebrated randomized clinical trials for many years are now recognising the pitfalls of focusing purely on this source.

A famous quote from the Scottish Intercollegiate Guidelines Network captures the new direction: “The accepted grading system was designed for application of efficacy; however, for medical practice settings, the RCT may not be practical, nor may it provide the best evidence. Guideline users may misinterpret the grade of recommendation or they may fail to properly weigh lower grade recommendations”.

Different designs are possible but many of them have limitations and there are pros and cons that should not be underestimated or over pronounced. There is a lack of guidelines for real life trials and this is something that needs to be worked on in order to craft a more reliable framework and share experiences that move us ahead in guiding real life studies. And finally, in light of these huge uncertainties, there is a great need for transparency relating to the use and the usefulness of real life data provided to reimbursement authorities today.

Some Case Studies
Below, we discuss cases related to five fairly routine uses of real life data:

1. Patient profiling and prevalence
Information on patient numbers and patient characteristics is increasingly required to guide decisions on targeting and appropriate treatment - important to many countries whether they have cost effectiveness as a criteria or some kind of technology assessment and appropriate use criteria. This is generally driven by the belief that appropriate targeting provides the best cost effectiveness. Patient profiling enables models to be localised or tailored. Take the example of Attention Deficit Hyperactivity Disorder. This is treated very differently in different countries, which means the relevance of comparators and where they are used in the treatment sequence varies. Record capture can help to provide this insight. Table 1 shows the kind of external validity that can be obtained with the use record data - in this case drawn from the IMS disease analyzer dataset and compared to a large, classic epidemiology type study in gout.

Patient profiling enables models to be localised or tailored. Take the example of Attention Deficit Hyperactivity Disorder. This is treated very differently in different countries, which means the relevance of comparators and where they are used in the treatment sequence varies. Record capture can help to provide this insight.

Table 1 shows the kind of external validity that can be obtained with the use record data - in this case drawn from the IMS disease analyzer dataset and compared to a large, classic epidemiology type study in gout.

A reasonable level of agreement can be seen in some of the data, alongside the finding that some things were captured in routine records that had not been addressed in the study questionnaire per se.

Patient prevalence lies at the heart of understanding market size, patient size and funding issues. Representative sampled databases can provide statistical estimates of the treated prevalence and complement published epidemiology. Registries often have a centre or regional focus. Also, patient subtypes can be assessed whereas published epidemiology tends to focus on general conditions.

2. Treatment flows The use of treatment flows is well established in HEOR and models can be built with more relevance and less reliance upon expert opinion. Figure 3 illustrates an anonymized example for a cardiovascular indication, showing the ability to map and better understand the pathway of treatment and patterns of care.

It can be seen that the majority of new patients presenting with this CV indication are treated with Drug 1. Most of the patients placed on this drug are switched, predominantly to Drug 5 and a minority remained untreated.


Retrospective Cohorts
Retrospective Cohorts offer a very valuable role because the idea of longitudinal data sets is some distance off in all areas. What can be done? The kind of information most valuable obtained relates to case records right back to diagnoses, generating information on treatment algorithms, resources, timings between therapies, and information about the level of disease control and treatment response. This can apply to primary or secondary care physician records. While there are some routinely syndicated products, of absolute most value is where information in these can be tailored or ‘topped-up'. Where data are collected on a cross-section basis, it is possible to go in and collect additional data, get extra questions in the core question set. This is very valuable because there will not be a core questionnaire that will answer all questions - it’s just not possible.

3. Compliance and Persistence This examines the extent to which a medication is taken as prescribed and the extent to which adherence to the treatment course occurs. In the case of depression and schizophrenia, lack of persistence and/or compliance is absolutely key as to whether disease control and prevention of relapse occurs. So this is not a kind of an academic exercise - it is what actually drives disease control - and resource use - in practice. Schizophrenia and depression are both particularly expensive to treat when it comes to institutionalization which makes these types of estimates key. In the case of diabetes and heart disease, the aim is clearly to halt the natural progression of the disease by controlling the key biology in each case. Whether for economic models or not, these insights can help to differentiate the value in real life.

The example below shows some fairly sophisticated statistical modeling used in Cox Regression and proportional hazards modeling which can accommodate multivariate analysis.

Here, in relation to anti-hypertensives in the UK, it was found that ARBs and diuretics were associated with better persistence. The application of a good multivariate technique enabled a better understanding of the underlying dynamics. Seven factors independently increased this persistence. On the plus side, the driver of relatively greater persistence was men with a recent MI. These men are perhaps quite scared and so stayed with the therapy while on the other side of the fence, people who have had a diagnosis some time ago and people who had a MI some time ago, had a greater relative risk to drop off.


4. Treatment costs and costs in disease stages and disease states Treatment costs can be estimated directly, either linked to disease or in stages specifically which provides considerable power when it comes to populating models and populating arguments about cost. Often drug treatment cost is available from databases, but typically in Europe, the ability to extract cost information is lacking. What we tend to see is resource use information and this can be costed in a secondary way. In terms of some findings about cost of treatment in Parkinson’s disease, the example below shows the differences across age and between countries. In Germany, treatment is split between neurologists and primary care physicians; from the UK came the interesting phenomenon of more money being spent in younger age groups.

5. Health outcomes and disease sequelae This is without doubt the most challenging aspect. Economic evaluation itself has a preferred time horizon which is often a long-term horizon. Chronic disease evaluation means the same thing - long-term. Disease sequelae gaps exist - often this type of information cannot be found in trials. In the case of metastatic colorectal cancer, treatments can enhance the ability to recess secondary tumors of the liver. The survival rate for those who have had a recession versus those who have not is a very powerful piece of information and is something that will not have been available in a trial. Comparative health outcomes are the key and the biggest challenge. A further case study provides an example of disease control in gout - looking at the level of disease control as defined by flare-ups, either one flare-up or two or more according to various sub-groups. Interestingly, this shows that German patients were, for some reason, better controlled.

  Issues Index | 2007 Issues Index