OBJECTIVES: The purpose of this study is to further examine potential mode effect on item difficulty hierarchy by taking item response theory (IRT) approach. Specific objective is to examine if the Stroke Impact Scale-16 (SIS-16) items function in the same way between the two administration modes in item difficulty hierarchy. METHODS: Patients with stroke were identified using ICD-9CM in 13 participating Veterans Affairs (VA) hospitals in administrative database and confirmed by reviewing electronic medical records. Patients were randomized to mail or telephone groups to administer SIS-16 at 12-weeks after stroke. Baseline demographics and severity of stroke were compared. Item difficulties and errors associated with the difficulties were estimated with the Rasch model using Winsteps. RESULTS: Telephone and mail group are composed of 96 and 92 patients, who were comparable in terms of demographics and stroke severity. SIS-16 scores distributed without severe ceiling or floor effects. The ‘walk fast' was the most difficult and ‘stay sitting without losing balance' was the easiest item. The item difficulty hierarchy was maintained in 10 items, but three items, carry heavy object with affected hand, dress the top part, move from bed to chair, were as more difficult in telephone survey while reported walk one block, control bladder, get to the toilet were more difficult in the mail administration group. However, considering the error associated with the difficulty showed that all 16 items were within 95% confidence intervals, thereby failing to show significant Differential Item Functioning (DIF). CONCLUSION: While the items of the SIS-16 showed some discordance in item hierarchy across mail and telephone administrations, none of the items showed significant DIF. These findings suggest that the mode of administration does not influence item difficulty hierarchies. As technologies such as internet and emails are more accessible, a research agenda needs to be developed to facilitate efficient follow-up.