FDA Perspective: What is FDA’s Experience with Post Approval Development Studies? What is the Primary Evidentiary Role of these Studies?

The Official News & Technical Journal Of The International Society For Pharmacoeconomics And Outcomes Research

The Application of Post-Market Registries and other Evidence for Medical Devices - Part I: FDA Perspective: What is FDA’s Experience with Post Approval Development Studies? What is the Primary Evidentiary Role of these Studies?

Neal Muni, MD, MSPH, Medical Officer*, US Food and Drug Administration, Rockville, MD, USA

This is a summary of the presentation given at the ISPOR Medical Device and Diagnostics Council Forum at the ISPOR 10th Annual International Meeting, May 17, 200, Washington, DC, USA.

Device versus Drug Therapies: Implications for Post-Marketing Device Evaluation
When developing the proper clinical trial studies for medical devices and diagnostics, there are limitations for both pre- as well as post-market studies. The ability to blind treatments for many types of medical devices is very difficult, logistically as well as physically. Medical device trials are more often single-arm, non-blinded studies, which is in contrast to drug studies, which are predominantly randomized, controlled trials. Scientifically, a randomized, controlled, blinded trial is preferable for device trials, to ensure equal allocation of patient cohorts to investigational and control therapies, and to minimize biases inherent in nonrandomized selection of patients to particular treatments. However, in many device trials, such a trial design is not possible. Some differences between device and drug development include the following:

Developmental Feature	Device	Drug
Rate of technology change	High	Low
Influence of Physician technique	High	Low
Population size	small	large
Ease of in vitro assessment	High	Low
Ability to visualize performance	High	Low
Definition of “orphan”	4,000	200,000
Pivotal studies required	1	2
Ability to blind treatments	difficult	easy

The average product lifecycle for medical devices is measured on the scale of months as contrasted with drugs, in which product lifecycles are usually measured in years. Any type of study (pre-market and more importantly post-market) must take into account the device life cycle. One key difference between device and drug therapies is the concept of incremental design changes to devices, which can be well-characterized based on engineering and other laboratory bench testing, in contrast with small molecule therapeutics, in which structural molecular changes could completely modify the pharmacologic and toxicologic properties of the drug. If a device modification can be fully characterized by in vitro and pre-clinical (animal) testing, substantial clinical testing of the modified product might not be necessary. In addition, post-market studies for various iterations of devices may be different in scope compared to those necessary for drug therapies.

In the device life cycle, all phases of development should be considered as a continuum of risk/benefit evaluation, including the design phase / pre-clinical evaluation, clinical testing phase (pre-approval), post-marketing phase, and iterative technological improvements / obsolescence of earlier versions. This requires a more integrated approach balancing the pre- and post-approval phases.

Post-marketing medical device evaluation tools include the following:
• Sentinel Reporting Centers “MedSun”
• MDR (Medical Device Reporting) System
• Post-approval Studies
• Post-marketing Surveillance “Section 522” Studies
• Registry Data Evaluation

Sentinel Reporting Centers The FDA contracts with outside medical centers to provide data on medical devices. This medical surveillance network, the medical device surveillance network or MedSun, helps the Agency obtain quality data, clinical information and more demographic information on devices that many times are not detailed in medical device report (MDR) data.

Medical Device Reporting System The largest and substantial amount of data we do get postmarket comes from Medical Device Reporting System (MDR), a mandatory, but passive reporting system. The MDR is not a study in itself, but rather, a collection of data for medical devices, secondary to adverse events. It is mandated for the manufacturer and healthcare settings to provide this data. MDR serves as the “default” post-approval method of surveillance.

Post-approval Studies The FDA can request post-approval studies, which are conditions of approval studies mandated by the FDA. Because of the rapid life cycle of devices, postapproval studies can be requested earlier in the approval process. Recently, the FDA Division of Post-Market Surveillance has taken over the authority and review of these types of studies.

Post-marketing Surveillance “Section 522” Studies For targeted assessments, which have very specific criteria for implementation, the FDA has the authority, “Section 522”, to mandate a study subsequent to the approval of the device. A section 522 study must be focused on a specific question to be answered, and cannot be used for more global, open-ended assessments of product safety and effectiveness.

Patient Registries Registry studies are very heterogeneous study designs with varying levels of scientific rigor. They may include “informal” data collection of safety and performance information without any entry criteria or defined follow-up. They may be more formalized studies with inclusion/exclusion criteria, case report forms, and defined follow-up intervals. The scientific validity of such data may be very limited in its utility to make any clinical and regulatory inferences. It is very important to make sure that, when the term “registry” is used, it is clear what is being discussed. Registries could be an additional step forward in product evaluation, if one formalizes inclusion/ exclusion criteria, has a rigorous way of collecting information on defined case reportforms, a rigorous auditing system to audit the data collection, defined follow-up intervals, an infrastructure to make sure that there are minimal losses to follow-up in the tracking system. It is realized that these registries start to lose their characteristics as “registries” and become defined studies, but can still be considered registry studies. Registry data is a source of information for which the FDA is still gaining experience for medical devices. Potential sources of these registry data can include academic or other types of centers and other third parties. Examples include professional society databases such as the American College of Cardiology and AdvaMed-sponsored databases. For existing databases, the Agency needs to be assured a proper infrastructure to complete these studies, as well as assurance of the scientific validity of these studies.

Post-Approval Data Collection Methods
The different types of post-approval data collection methods, ordered in terms of the scientific rigor & validity, include the following:
• Randomized, double-blinded, controlled trial
• Randomized, unblinded clinical study
• Observational (conditions of approval) studies.
• Non-randomized registry study with formal follow-up and data collection (single arm trial)
• Informal registry study (“open enrollment”) with less stringent follow-up and data collection
• Sentinel reporting centers (“MedSun” contracted clinical centers)
• Medical Device Reporting (MDR) system
• Tabulation of adverse events reported to product manufacturer
• Product complaint reports sent to manufacturer
• Voluntary reports by healthcare professionals/users

All of these studies can provide information on the product’s performance in the ‘real world’. However, this information must be put in the proper context, in terms of the limitations and the strengths of the data. Post-Market Regulation Post-market studies provide challenging regulatory issues for the FDA. There are limitations to pre-approval studies, especially for devices, given their short product lifecycles and smaller populations compared to drug therapies. For pre-approval studies, clinical trials are performed

Post-Market Regulation
Post-market studies provide challenging regulatory issues for the FDA. There are limitations to pre-approval studies, especially for devices, given their short product lifecycles and smaller populations compared to drug therapies. For pre-approval studies, clinical trials are performed in well-defined populations and attempt to be representative, but these studies may not adequately evaluate the product in special population subsets and lack ‘real world’ use data. The “catch 22” is that we only know the true performance of a product after approval, but the product must be safe and effective in order to be approved. There are limitations to existing post-market tools such as the MDR passive reporting system and registry studies, which must be considered when interpreting data from these sources. In addition, trends or signals regarding product safety concerns may only be identified or better understood in retrospect.

A major regulatory challenge is to find the appropriate pre and post-approval balance for obtaining clinical data. On the one hand, a substantial data requirement as a precondition for marketing can prolong the approval process and delay product availability. The device might fill a significant unmet need or offer significant improvement compared to existing therapies. On the other hand, approving insufficiently tested therapies could have disastrous public health safety consequences, especially for breakthrough therapies that are rapidly and widely adopted in use, especially in certain populations that were not adequately tested in pre-market studies. It is a difficult balance, especially with the tools available for postmarket data collection. The MDR is a passive reporting system with no “denominator” information, and limited clinical information reported. FDA’s dilemma is timing and nature of regulatory action if a “signal” is identified. Analysis of structured post-approval study data might not be available at time of initial MDR reports. The implication of inaction or delayed action is a very serious concern.

In conclusion, from the FDA perspective, post-marketing data collection for device evaluation is an essential component in the assessment of device performance. Post-marketing studies provide information regarding ‘real world’ use; they provide information patient subsets not fully tested in pre-market clinical trials, and they can give a confirmation of data observed in pre-approval studies. However, post-market evaluation of medical devices presents many challenges, including the limitations of existing tools and the need to enhance capabilities of existing tools. Product risk and benefit must be assessed across all stages of the product life cycle.

Issues Index | 2005 Issues Index