Objectives: Scale development and evaluation is particularly challenging in rare diseases, due to small samples and heterogenous clinical profiles. Recent recommendations support a flexible approach which includes multiple sources, iterative mixed methods and repeated measurements to estimate the psychometric characteristics of an instrument when sample size is small. However, detailed methodological accounts of COA examination in rare disease are still rare. This study describes an iterative mixed method approach for the evaluation, modification and measurement maximization of the INAD-RS, a 40-item ClinRO developed to assess the disease progression in children with Infantile Neuroaxonal Dystrophy (INAD).

Method: The study design included qualitative interviews (5 healthcare professionals and 10 caregivers of children with INAD), independent experts’ feedback, and secondary analyses of evidence gathered as part of a National History (NH) study conducted in Tunisia, China, and India. In addition, independent video reviews of daily activities of 17 children, exploratory Classical Test Theory (CTT) and Rasch Measurement Theory (RMT) analysis of 39 baseline assessments were also conducted. A second evaluation was conducted with 57 baseline data from the NH and 11 longitudinal assessments gathered from an open label treatment study.

Results: Results of the interviews, video analysis, experts’ feedback, and the initial exploratory CTT and RMT highlighted for investigation problematic items and scoring issues, possibly related to developmental and regional differences. The original 40 item questionnaire was modified to 26-item scale (mINAD-RS26). The second CTT and RMT analyses provided evidence to support measurement performance.

Conclusion: The mINAD-RS26 was carefully evaluated using an iterative mixed method approach that aligns to the best practices in scale development and evaluation, especially for a rare, ultra-orphan disease. Results were promising given the caveats.