OBJECTIVES: Calibration ensures that models' predictions closely match real-world observed outcomes. However, calibration target data are often imperfect—noisy, imprecise, outdated, or biased. This study presents a regression-based method for quantifying two value-of-information metrics: eliminating the uncertainty in calibration target data (expected value of partial perfect information, EVPPI) and collecting better calibration target data (expected value of sample information, EVSI).

METHODS: We used a simple three-parameter model to predict the net benefit (NB) and a hypothetical epidemiological output. Each model output was a linear function of a distinct pair of the three parameters. The epidemiological output was used to calibrate the model. One million samples were drawn from the joint posterior distribution of the calibration parameters and used to perform a probabilistic sensitivity analysis (PSA). Two regression models were fitted to the NBs of the PSA samples using ordinary least squares (OLS) and generalized additive models (GAM). We estimated the EVPPI from the first model, which regressed the NBs against the corresponding epidemiological outputs. The EVSI was calculated from the second model, which regressed each NB against the summary statistic of a dataset sampled from a normal distribution with a mean equal to the corresponding epidemiological output. The regression-based results were compared to the closed-form solutions.

RESULTS: The analytical solutions for the EVPPI and EVSI were £0.132 and £0.104, respectively. The estimated values (standard error) for the EVPPI were similar using OLS and GAM, at £0.130 (0.0018). For the EVSI, the calculated values were comparable using OLS, at £0.104 (0.0017), and GAM, at £0.106 (0.0017). These results demonstrate that both regression models closely approximate the analytical values.

CONCLUSIONS: These results highlight the accuracy of regression methods in estimating the EVPPI and EVSI of calibration target data. Quantifying the decision uncertainty associated with calibration target data enables better-informed resource allocation decisions.