OBJECTIVES: As the patient perspective becomes increasingly critical in this era of patient-centric care and patient cost-sharing, a better understanding of research methodologies and their impact on validity of responses is required to assess the growing body of patient research. Increasingly, researchers are able to leverage technology in order to access patient panels; however, questions remain as to their robustness and representativeness. Here, we review the underlying survey response differences obtained from both nonprobability and probability-based patient samples accessed via online survey panels and the potential impact to patient-centered research. METHODS: A review was conducted of a mixed-method patient survey examining self-identified smokers and motivation to quit, which utilized both probability and nonprobability samples. Survey results were compared from 1,427 probability and 5,425 nonprobability patient samples. The probability sample was obtained from a statistically valid online panel, representative of the US population. Survey responses between groups were compared using Student’s t-test and Chi-square tests. RESULTS: The probability sample reported smoking more cigarettes per day (p<0.001) and were more likely to report smoking every day (p<0.001). Additionally, patients in the nonprobability sample were more likely to report that they planned to quit smoking in the next 30 days (p<0.001) and that they had made an attempt to quit smoking in the past 6 months (p<0.001). CONCLUSIONS: Survey results for patient reported outcomes studies obtained through nonprobability sampling varied significantly from those obtained through probability sampling; and response variation is unpredictable. Calibrating non-probability samples with probability samples using specific screening questions may minimize bias in the resulting estimates in the larger combined sample.  In order for patient-centered outcomes research to play a meaningful role in health care policy and decision making, rigorous probability sampling methodologies must be applied in order to allow interpretation of results with high confidence.