ESTIMATING SAMPLE SIZE FOR PSYCHOMETRIC STUDIES USING CONFIRMATORY FACTOR ANALYSIS

Author(s)

Cole JC, Cheng RCovance Market Access Services, Inc., San Diego, CA, USA

OBJECTIVES: Sample size determination is critical in planning psychometric studies in order to achieve stable results.  Confirmatory factor analysis (CFA) is an essential aspect of content validation and for psychometric studies.  The current study reviewed three approaches for CFA-based power analyses: N:q rule-of-thumb (N:q), RMSEA-based sample size (RMSEA-BSS), and Monte Carlo simulation (MC). METHODS: N:q states 10 (reliable loadings) or 20 participants (less reliable loadings) are required for every free parameter.  RMSEA-BSS uses the sampling distribution of RMSEA to determine the power of a CFA for a predicted level of RMSEA versus a given RMSEA criterion (similar to the power of a t-test for one mean versus another).  the MC is used to investigate the performance of statistical estimators under various conditions wherein data are generated for over k>10,000 simulated datasets to understand the sampling distributions of various parameters.  MC has many benefits, including statistical estimates of the sample size requirements for a very specific CFA model.  These three approaches were examined on the CFA for a common 10-item depression instrument. RESULTS: For n:Q, a total of 22 free parameters were present, thereby requiring between 220 or 440 subjects.  RMSEA-BSS was calculated using a free website (people.ku.edu/~preacher/rmsea/rmsea.htm), resulting in a sample size estimate of 283 (alpha=0.05, CFA df=31, power=0.8, RMSEA criterion=0.06, and RMSEA estimate=0.02).  A sample size of 183 was calculated using MC.   CONCLUSIONS: N:q is simple yet the least accurate technique used.  The RMSEA-BSS approach is not as accurate nor as flexible as MC, nor as quick as N:q. It is accessible to most psychometricians, however, and often provides sufficient accuracy.  The MC is sophisticated and highly accurate; it can include almost any latent modeling variant, thereby allowing for excellent specificity, its only downside is its complexity.  The current CFA in MC provided a marked savings in sample size.

Conference/Value in Health Info

2011-05, ISPOR 2011, Baltimore, MD, USA

Value in Health, Vol. 14, No. 3 (May 2011)

Code

PMH83

Topic

Methodological & Statistical Research, Patient-Centered Research

Topic Subcategory

Confounding, Selection Bias Correction, Causal Inference, Modeling and simulation, Patient-reported Outcomes & Quality of Life Outcomes

Disease

Mental Health

Explore Related HEOR by Topic


Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on ispor.org. Update my browser now

×