ESTIMATING SAMPLE SIZE FOR PSYCHOMETRIC STUDIES USING CONFIRMATORY FACTOR ANALYSIS
Author(s)
Cole JC, Cheng RCovance Market Access Services, Inc., San Diego, CA, USA
Presentation Documents
OBJECTIVES: Sample size determination is critical in planning psychometric studies in order to achieve stable results. Confirmatory factor analysis (CFA) is an essential aspect of content validation and for psychometric studies. The current study reviewed three approaches for CFA-based power analyses: N:q rule-of-thumb (N:q), RMSEA-based sample size (RMSEA-BSS), and Monte Carlo simulation (MC). METHODS: N:q states 10 (reliable loadings) or 20 participants (less reliable loadings) are required for every free parameter. RMSEA-BSS uses the sampling distribution of RMSEA to determine the power of a CFA for a predicted level of RMSEA versus a given RMSEA criterion (similar to the power of a t-test for one mean versus another). the MC is used to investigate the performance of statistical estimators under various conditions wherein data are generated for over k>10,000 simulated datasets to understand the sampling distributions of various parameters. MC has many benefits, including statistical estimates of the sample size requirements for a very specific CFA model. These three approaches were examined on the CFA for a common 10-item depression instrument. RESULTS: For n:Q, a total of 22 free parameters were present, thereby requiring between 220 or 440 subjects. RMSEA-BSS was calculated using a free website (people.ku.edu/~preacher/rmsea/rmsea.htm), resulting in a sample size estimate of 283 (alpha=0.05, CFA df=31, power=0.8, RMSEA criterion=0.06, and RMSEA estimate=0.02). A sample size of 183 was calculated using MC. CONCLUSIONS: N:q is simple yet the least accurate technique used. The RMSEA-BSS approach is not as accurate nor as flexible as MC, nor as quick as N:q. It is accessible to most psychometricians, however, and often provides sufficient accuracy. The MC is sophisticated and highly accurate; it can include almost any latent modeling variant, thereby allowing for excellent specificity, its only downside is its complexity. The current CFA in MC provided a marked savings in sample size.
Conference/Value in Health Info
2011-05, ISPOR 2011, Baltimore, MD, USA
Value in Health, Vol. 14, No. 3 (May 2011)
Code
PMH83
Topic
Methodological & Statistical Research, Patient-Centered Research
Topic Subcategory
Confounding, Selection Bias Correction, Causal Inference, Modeling and simulation, Patient-reported Outcomes & Quality of Life Outcomes
Disease
Mental Health