OBJECTIVES: To pilot test a best-worst scaling (BWS) survey designed to identify important barriers to coronavirus disease 2019 (COVID-19) vaccination among African Americans living in the Black Belt region.

METHODS: A cross-sectional, web-based survey using BWS Case 1 (the object case) was conducted among African Americans living in the Black Belt region, including a string of counties in 11 states. Thirty-two potential barriers to COVID-19 vaccination were identified from the literature. A nested balanced incomplete block design was used to generate 62 blocks of 16 choice tasks. Each choice task included six barriers. Participants were asked to choose the most and least important barriers in each choice task of a block. The survey included demographic information, COVID-19 experiences, and an open-ended question for the participants' opinions. The survey was pilot-tested with a total of 30 participants. The completion time and responses were examined. A simple best-minus-worst count and the natural logarithm of the square root of best counts divided by worst counts of each barrier were calculated to rank the importance of barriers.

RESULTS: Most participants were female (50%) and full-time employed (50%) with an average age of 34 years. The average survey completion time was approximately 12 minutes. The responses to the survey questions tended to be complete, and no important comment about the survey was found. The three most important barriers were “inconsistent information of COVID-19 vaccines,” “lack of time to get COVID-19 vaccine,” and “dosing inconvenience of COVID-19 vaccines.” On the other hand, the three least important barriers were “distrust of government agencies,” “safety concern of COVID-19 vaccines,” and “distrust of clinical studies of COVID-19 vaccines.”