|
|
A Checklist for Medication Compliance and Persistence Studies Using Retrospective Databases |
|||||||||||||||||||
|
Special Interest Group Chair:
Leadership Group:
This report was published in Value in Health as follows. Principles Of Good Research Practice: Report of the ISPOR Medication Compliance & Persistence Special Interest Group A Checklist for Medication Compliance and Persistence Studies Using Retrospective Databases Andrew M. Peterson PharmD1, David P. Nau, PhD2, Joyce A. Cramer, BS3, Josh Benner, PharmD, ScD4 , Femida Gwadry-Sridhar, PhD, RPh, MSc, BSc5, Michael Nichol, PhD6 1University of the Sciences in Philadelphia, Philadelphia, PA, USA; 2University of Michigan, Ann Arbor, MI, USA; 3 Yale University, West Haven, CT, USA; 4ValueMedics Research, LLC, Falls Church, VA, USA; 5 McMaster University, London, ON, Canada; 6 University of Southern California, Los Angeles, CA, USA; ABSTRACT PURPOSE OF THIS ARTICLE To help the readers and designers of such studies, the International Society of Pharmacoeconomics and Outcomes Research (ISPOR) charged the ISPOR Medication Compliance and Persistence Special Interest Group (SIG) to develop a checklist of items that should be either included, or at least considered, when a retrospective database analysis of medication compliance or persistence is undertaken. The Analytic Methods Working Group of this SIG met frequently to define the appropriate elements for such a checklist. The members of this working group consist of researchers, academicians, and practitioners with a record of publication and interest in medication C&P and retrospective database analysis. Some elements in this checklist were drawn from other ISPOR efforts related to retrospective databases and medication compliance. For example, the definitions of compliance and persistence were drawn from the ISPOR Compliance and Persistence SIG Definitions Working Group (http://www.ispor.org/sigs/medication.asp), and the broader discussion of retrospective database analyses was drawn from the Advisory Panel Report discussing methodological issues with retrospective and claims data studies [1]. Note that the terms "compliance" and "adherence" are considered synonyms, while both differ from "persistence." Explicit definitions of compliance and persistence should, however, be provided by the researcher for individual studies (as noted later in this article). FRAMEWORK OF CHECKLIST
HOW SHOULD THE CHECKLIST BE USED? Description of Checklist Elements Title The title of the study should be descriptive and reflect its purpose. Furthermore, the title should include the appropriate term(s) as represented by the study. This would include the retrospective nature of the study, the population(s) being examined, and the appropriate measure of compliance or persistence.Example. Title: "A retrospective analysis of medication persistence among children taking stimulant medications for the treatment of attention deficit hyperactivity disorder (ADHD)." Abstract The title should be a short description of the study. The abstract, presented at the beginning of an article, should be a short summary of the objectives, methods, results, and conclusions. Structured abstracts require the author to follow a specific format. The purpose of the structure is to provide a systematic means of organization. Some journal editors request that the abstract be "the paper in miniature," completely self-contained. The revised Consolidated Standards of Reporting Trials statement strongly encourages abstracts to be in a structured format to allow the reader to locate information more easily and potentially improve the quality of the abstract [2]. In this vein, the Methods section of the abstract should define the types of analyses used, and the Results section should describe the extent of the findings using those methods. The main results of the analyses should be stated numerically in the abstract. The Conclusion section should not overextrapolate the results and should only reflect the true findings of the study. Be aware that almost 5% of abstracts contain erroneous information [3]. Note that abstracts require great attention to accuracy because they are more widely available than full articles. Introduction Section The first part of the Introduction section should review the literature in the area of study. This scientific background should allow the reader to understand the rationale for the study being conducted and describe the nature of the problem or issue that the study intends to investigate. Furthermore, the scope and severity of the problem should be addressed using a clear review of the fundamental literature related to the topic being addressed, including appropriate clinical and health economic literature, along with the C&P literature. Objectives and Definitions The selection of an appropriate study objective is important because it drives the study design and variables being measured. These are important issues to address because the design and methods for the study should allow the researcher to measure appropriately the compliance or persistence variable and fulfill the objectives of the study. Therefore, in this section of a study, the reader should find clearly stated objectives and an indication of the primary outcome of interest. The author should indicate when compliance or persistence is the primary "outcome" of interest (the dependent variable), or being used as an explanatory or control variable to explain variance in an outcome. In either case, the author should provide explicit definitions of compliance or persistence based on a published definition with appropriate literature reference. Recommended definitions have been promulgated by the ISPOR Medication Compliance and Persistence Definitions Group and are available on the ISPOR Web site ( http://www.ispor.org/sigs/medication.asp ). Design and Methods This section is designed to help the reader/researcher focus on many of the key elements of a well-conducted retrospective database analysis involving medication compliance. The importance of linking the appropriate study design to the objectives and compliance or persistence measure is emphasized, as is the importance of clearly delineating the population being studied. Design The three major types of study designs are exploratory, descriptive, and explanatory. An exploratory study of medication compliance or persistence does not involve hypothesis-testing and is often qualitative in nature. Descriptive studies of medication C&P may employ qualitative or quantitative methods to describe the medication-use patterns of a population An explanatory study is designed to investigate the relationship between medication C&P and other variables. The design of the explanatory study is, however, crucial to explaining the casual nature of the relationships studied. Many database analyses employ a study design known as the historical cohort. This study design is nearly identical to a prospective cohort study, except that the data have already been collected and are usually stored in an electronic database. In this study design, the researcher constructs the cohort by selecting patients already treated, meeting certain inclusion/exclusion criteria. This design may facilitate an examination of the relationship between C&P and other variables but limits the researcher's ability to assess causation because of the limitations typically associated with retrospective studies (e.g., selection bias, missing or incomplete information, or censoring bias). The design of the study should be clearly stated and this design should match the objectives of the study. For example, a longitudinal study using a retrospective database may allow the researcher to determine the incidence of noncompliance, whereas a cross-sectional study only allows the researcher to determine the prevalence. Furthermore, a researcher should recognize the limitations of using retrospective data in establishing the causal relationship between C&P and other variables. Data Sources A well-described retrospective database analysis allows the reader to identify the population from which the sample is drawn. (For a full discussion of the key elements to defining the population and associated variables, the reader is referred to the ISPOR Advisory Panel Report on retrospective databases [1].) The methods for sampling need to be adequately described so the reader can infer whether the sample is representative of the population of interest. Also, a full description of the data source is important. The researcher should clearly indicate whether the data source is a public or a commercial database, or if it is not a prescription claims database (e.g., disease registry). The time frame for the data set needs to be described and the length of study should be clearly delineated so that an assessment of the appropriateness of the study period can be made in relation to the objectives. To this end, the researcher should clearly state the inclusion and exclusion criteria for the study and describe the rationale for these criteria. To ensure that the sample represents the population of interest, the method by which the researcher verified subjects meeting the inclusion/exclusion criteria must be present and appropriate. For example, the continuous eligibility for a prescription benefit during the study period should be verified to determine whether patients had sufficient data to make a valid estimate of compliance or persistence (i.e., patients need at least two fillings of a medication to calculate a medication possession ratio). Furthermore, there should be a description of the pre-enrollment period and a determination of whether a subject was truly naive to the drug, if important to the study. The investigators should describe how the subjects were identified, including a prestudy period to determine prestudy medication use and diagnoses. Other areas for consideration relate to the definitions used to select the subjects for inclusion. For example, did the researchers use diagnosis codes versus prescription claims to categorize patients as having diabetes? Also, if the researchers employed a matching process (if appropriate to the study design), did they describe it adequately and was it sufficient in detail to assure appropriate matching could occur? The purpose of the matching strategy is to minimize the potential for selection bias. The researcher also needs to assure the reader that every effort was taken to protect the confidentiality of subjects, such as Institutional Review Board/Ethics Committee approval or meeting Health Insurance Portability and Accountability Act guidelines (for US studies). Coupled with this is evidence that the data have been appropriately "cleaned" (entries that are clearly erroneous are eliminated or fixed) and that the researcher provided evidence for the reliability and accuracy of the data. The investigators should explain how cleaning/editing the data set affected eligibility, often by further excluding specific types of cases [1]. Measurement of Compliance The transparency of the measurement of C&P is very important. The researcher simply stating that the method of calculating C&P is "proprietary" and cannot be disclosed is not conducive to scientific dialogue. Therefore, the methods for calculating the C&P variable should be clearly described. Every effort should be made to use standard methods for calculating C&P so that it is possible to interpret the findings of the study in context with other studies. It is important to note that the measure chosen as the C&P variable should be consistent with the objective of the study. For example, researchers interested in measuring compliance rates should not use a variable that actually measures persistence, or vice versa. There are several methods used to calculate C&P. The following are just a few. (The reader is also referred to Steiner et al. [4] or Farmer [5] for more examples.) Measures of compliance. A variety of methods are used to estimate a patient's compliance using retrospective databases. One of the most common methods is to calculate the medication possession ratio (MPR). When this ratio is calculated across multiple refills, it may also be called the continuous measure of adherence (CMA) [6]. These measures are typically calculated using the basic formula noted below: Number of Days of Medication Supplied within the Refill Interval/Number of Days in Refill Interval This is usually calculated by summing the number of days supplied for all but the last refill, divided by the number of days between the first and the last refill. Therefore, at least two fill dates are required to calculate this ratio. Researchers, however, may choose a fixed time frame for the refill interval rather than using the last refill as the end point for the refill interval. Within most US-based prescription claims databases, the "days supply" is usually included as a data field within each prescription claim (e.g., 60 tablets of a medication that is taken twice daily would yield a 30-day supply), along with the dates that the prescription was filled or refilled. In some non-US databases, the researcher may need to estimate the days supply for each drug by applying the defined daily dose to the quantity dispensed. Other methods include a continuous measure of medication gaps (CMG), in which the sum of the days in the gaps between refills in the observation period is divided by time between the first and last fills. This estimate provides an indication of the percentage of time the patient does not have the medication available for use. For example, the filling behavior of a cohort of patients being treated for congestive heart failure may be highly variable, resulting in numerous gaps in digoxin use. In some cases, the gaps can be negative (early fill) as well as positive (late fill). The CMG would require the analyst to add the positive and negative gaps for the period of observation. This measure provides an indication of the variability in refill behavior. An example of a study that used CMG in diabetes patients is Morningstar et al. [7]. Measure of Persistence Persistence adds the dimension of time to the analysis and usually represents the time over which a patient continues to fill a prescription, or the time from the initial filling of the prescription until the patient discontinues refilling a prescription. The most common time unit is days, but could also be months or years. One means of calculating this is the estimated level of persistence (ELPT) method [8]. This calculation (below) allows the researcher to determine the percentage of individuals remaining on therapy (persistent) at a given time. ELPT may differentiate patients taking a medication sporadically during a defined time frame from those patients stopping the medication early during the same time frame [8]. The data can be displayed on a persistency curve, very similar to a Kaplan–Meier curve. The most common analysis is a Kaplan–Meier life table with discontinuation considered as elimination. Proportion of patients refilling each subsequent prescription with (X*days supplied) from fill. Dezii used this measure to help differentiate patients taking a medication sporadically during a defined time frame versus those patients stopping the medication early during the same time frame. Proportion of days covered. The proportion of days covered (PDC) is a measure of patient compliance that has been used with increasing frequency [9–13]. The PDC is calculated as the number of days with drug on-hand divided by the number of days in the specified time interval. The PDC may be multiplied by 100 to yield a percentage. The numerator of the PDC is not merely a sum of the "days supplied" by all prescriptions filled during the period. Rather, filled prescriptions are evaluated using a set of rules to avoid double-counting covered days. Thus, the PDC is always a value between 0 and 1. The denominator for the PDC is typically a clinically meaningful number of days that is the same for all intervals and patients (e.g., 90 days). The PDC can be analyzed as a continuous measure or divided into categories for use as an ordinal or dichotomous variable. When measured repeatedly and analyzed using appropriate statistical methods for within-subject repeated measures, the PDC has the advantage of simultaneously reflecting both C&P. Data based on this approach are frequently described in a figure to illustrate time trends. Other measures of persistence. Alternative analyses include number of days to discontinuation and number of prescription refills over a period of time. The days to discontinuation is a simple count of days from the index prescription to the date of the final dispensing, although some researchers include the days for which the final fill provided dosing (e.g., final 30 days). The number of refills, usually within 12 months of the index fill, could include patients who refill long after the allowed 30- or 60-day gap for being considered nonpersistent. This is a valuable calculation for drugs that may be used "as needed" without detriment to the clinical condition (e.g., treatments for seasonal allergy). Measurement issues. The researcher needs to account for the how anomalous values were handled. Some measures of C&P allow for the calculation of "hyper-compliant" values (e.g., MPR > 1 or negative gaps). The researcher should describe how and why these values were incorporated into the analysis. If an atypical method is used for calculating compliance, the researcher should report the rationale for the new method along with the formula for its calculation. Similarly, when the researcher collapses multiple medications into a single compliance estimate, the rationale and formula for this variable should be included. Examples of this are when the average MPR or gap across different medications is used to estimate overall compliance. If the researcher uses this strategy, then the analysis should also control for the influence that varying numbers of medications can have on the compliance variable itself. The Methods section also should explain how the analysis handled patients who switched to another medication in the same class (e.g., another phenothiazine) or one that is used for the same diagnosis (e.g., an atypical antipsychotic) [14]. For example, did the researcher drop those patients who switched drugs within the same class, or did the researcher estimate the "equivalent" dose of the two drugs and allow the patient to remain in the analysis. Furthermore, there should be some estimation of whether the method used for handling this variable was appropriate for fulfilling the study objective. Some researchers have categorized a drug-therapy switch as nonpersistence because it involves discontinuation of the initially selected drug. Nevertheless, because the term "persistence" is typically used to describe a patient's behavior, referring to a treatment switch as nonpersistence may suggest that the patient failed to take the product as directed even though the patient followed the directions appropriately. If the drug is not an oral solid dosage form (e.g., capsules or tablets), alternative methods are needed. For example, the dose of liquid, powdered, injected, and inhaled drugs may be prescribed in a way that leaves the patient with an under- or oversupply of drug at the end of the month. An attempt should be made to calculate an adjustment for wastage from the dispensing container, particularly for inhaled and injected drugs. For example, insulin wastage was estimated to account for higher than expected compliance in one study [15]. Statistical Analyses The distribution of the dependent variable must be considered when selecting an appropriate statistical test. The researcher should determine the best type of statistical test based on the type of data and their distribution. In general, parametric tests are preferred, but if the assumptions underlying a specific parametric test are violated, then a nonparametric equivalent should be employed. Nonparametric tests should be employed when • the data are measured and/or analyzed using a nominal or ordinal scale of measurement; • the probability distribution of the statistic is not normally distributed. When there is a cap on the MPR (e.g., maximum = 1), there may be a violation of the normality assumption, and therefore a nonparametric test should be considered. Similarly, if all the gaps in a gap analysis are converted to "no gaps" or "0," this also may violate the normality assumption. When subjects are categorized as "compliant" and "noncompliant" (e.g., when using a cutoff within the MPR to create the categories), a nonparametric test should also be used. In general, it is not wise to convert continuous data to categorical data. Statistically, there is a loss of power because of a decrease in the number of degrees of freedom (ANOVA models). Conceptually, dividing the data into arbitrary categories limits the utility of the information. It may be appropriate to use categorical data for a logistic regression, but with caution about the definition. If continuous data are converted to categorical data, the rationale for selection of cut-points should be provided and consistent with existing evidence for compliance in the selected population. The point at which discrimination is made for categorical definitions of compliance versus noncompliance should have been determined with a sensitivity analysis. An adequate discrimination of a cut-point has been made for few medications. One example is the need to take more than 95% of antiretroviral medication doses to avoid the development of resistance [16]. Few other investigations have looked at outcomes above and below the postulated categories. Researchers should be careful when using any categorical cut-point (commonly listed as 80%) unless they can document the clinical validity of the number as well as determine that that lower and higher values differ (sensitivity test). Selection of a cut-point usually requires information that patients taking more than this amount of medication have a clinically better outcome than patients taking less. This is not a statistical test but a clinical test of relevance. There are very few medications for which a cut-point has been determined. If the researcher makes multiple comparisons using the same data, there should be an adjustment made to maintain the experiment-wise alpha error at the prespecified level. Examples of appropriate adjustments include Bonferroni adjustment for multiple comparisons or the use of a post hoc test (e.g., least significant difference, Tukey's, Duncan's, Scheffe's) after determining a significant F-value in an ANOVA. Realistic power and/or sample size calculations should be described. If the researcher did not achieve the prespecified sample size, a recalculation of the power based on the actual results might be appropriate. There is controversy, however, regarding this approach. Others suggest that confidence intervals be calculated so that the readers can interpret the results on their own [17]. Furthermore, the researchers should make an attempt to control for bias in the data set. Bias can come in a variety of forms, including selection bias and measurement bias. The research should address how the potential for bias was handled. For example, propensity scoring is a technique used to control for systematic differences between groups by reducing the differences between groups to a single variable [18]. The researchers should explain the variables they chose to generate the propensity score and the results of the scoring. The researchers should indicate whether the covariates used in the process were balanced between the two groups, and if not, what steps they took to produce balance. If the researcher is evaluating an association between compliance and another variable, he or she should attempt to control for other variables that may confound the association being studied. To determine which variables directly affect compliance and which variables have mediating effects, the author should consider statistical techniques that facilitate answering the question. Such techniques include multivariate regression. To establish mediation, each variable is regressed in a hierarchical forward stepwise fashion on all other variables that precede it in the causal chain. The risk of misleading results increases as the ratio of independent variables to the number of patients increases. Therefore, the researcher should consider what variables act as confounders and which variables are covariates and should be controlled for accordingly in an analysis. Typical variables that may confound the measure of compliance include cost of medication, comorbidities, severity of illness (as measured by the Charlson comorbidity index or the chronic disease score), sex, and other sociodemographic factors [19,20]. Before adjusting for any variables, it is suggested that the researcher undertake a thorough review of the literature to establish variables that are known confounders within the disease state being studied and which covariates may show trends but are not known confounders. Establishing this a priori will also help guide the analytical plan. The researcher should attempt to maintain an alpha at 0.05, recognizing that often we have to deal with small sample sizes in our studies and that sometimes the analysis is exploratory in nature. Presentation and Discussion of Findings Results
|
|
Contact ISPOR @
info@ispor.org
| View Legal Disclaimer |