Session (CTI)

Purpose: This session is to advance COA measurement practice by addressing whether Cronbach’s alpha is fit for purpose and by equipping researchers with modern methods for evaluating reliability. As regulators increasingly request evidence beyond alpha to support an outcome measure’s fit-for-purpose, COA teams need clear guidance on when alpha is appropriate, when it is misleading, and how alternatives such as omega, ICC, split-half reliability, and IRT-based conditional reliability can strengthen COA submissions. This session will provide foundational understanding, practical analytic demonstrations, and regulatory context to help attendees improve reliability evaluation in clinical trials, qualification programs, and labeling claims. Real-time polling on current use of methods will have facilitate discussion. Description: Cronbach’s alpha remains the dominant measure of reliability in COA research, yet it is often applied under conditions that may violate its central assumptions or is used to support incorrect inferences. The session begins by clarifying the assumptions and properties of alpha, such as tau-equivalence, including unidimensionality and equal item contributions, and scale length dependence. The second presentation introduces contemporary alternatives, including McDonald’s omega, ICCs, split-half permutation methods, and IRT-based conditional reliability, explaining how each aligns with COA conceptual frameworks and different instrument designs. A third presentation compares alpha against these modern indices, using empirical examples, to demonstrate how reliability conclusions may change when multidimensionality, item information, or temporal stability are considered. The final presentation connects these methodological insights to regulatory expectations, highlighting how FDA and EMA increasingly critically evaluate internal consistency evidence, request domain-level reliability, caution against inflated alpha values, and rely on structural validity assessments.