HOW LOW CAN WE GO?--MAKING MEANINGFUL INFERENCES FROM SMALL SAMPLES
Author(s)
Bakken DG*, Bond M KJT Group, Inc., Honeoye Falls, NY, USA
Presentation Documents
OBJECTIVES: To explore the impact of inferences from very samples on the outcome of management decisions. In many cases management has some prior belief about the states of nature. We explore the potential advantage of incorporating Bayesian inference to improve the confidence in managerial decisions based on small samples.. Traditionally, survey researchers reconcile differences between survey results and prior beliefs by citing the uncertainty reflected in the sampling error or looking for other explanatory factors (such as possible survey measurement error). The Bayesian approach integrates the different sources of information (i.e., prior belief and observed survey results) to arrive at the most probable estimate. In full realization, a Bayesian approach considers not just the probability that “truth” lies outside some range of values but seeks to estimate the probability of each of many possible hypotheses, given the data was that obtained. METHODS: Using responses to a choice-based conjoint exercise that was embedded in an online survey of approximately 700 individuals, we created a series of samples of different sizes using different restrictions to reflect the ways in which both probability and convenience samples might be generated. We drew multiples of ten random samples of 25, 50, 75, 100, 150, 225 and 450 from our “population” of 897 respondents, resulting in 70 individual samples. We estimated HB models for each sample (using Sawtooth Software’s CBC-HB program). RESULTS: Simulated choice probabilities--a key output of discrete choice models--stabilize across samples starting with n=75. For smaller samples, decision confidence can be increased using Bayesian inference and bootstrapping methods. CONCLUSIONS: Meaningful inferences---and hence decisions--can be made with smaller sample sizes by utilizing Bayesian inference and methods such as bootstrapping to better estimate the degree of uncertainty in the data.
Conference/Value in Health Info
2013-05, ISPOR 2013, New Orleans, LA, USA
Value in Health, Vol. 16, No. 3 (May 2013)
Code
PRM196
Topic
Methodological & Statistical Research
Topic Subcategory
Confounding, Selection Bias Correction, Causal Inference
Disease
Multiple Diseases