Presentation (CTI)

OBJECTIVES: Guidelines for health economic modeling recommend parameter estimates "based on the best available evidence, in consideration of the full array of available information" while recognizing "time and resource constraints." Generative AI (GenAI) could transform parameter estimation practices by automating evidence synthesis, but performance benchmarks are necessary to guide adoption. We hypothesize that resource constraints lead modelers to cite parameters from previous studies rather than synthesize comprehensive evidence. This study defines a framework for analyzing current parameter sourcing practices by tracing citation chains in health economic models, proposing a minimum performance metric for GenAI tools.
METHODS: We developed a systematic approach to document parameter sourcing practices, and deployed it for an exemplar parameter: probability of witnessed overdose. We identified all simulation modeling studies using this parameter through PubMed, extracted their citation patterns, and constructed citation networks.
RESULTS: For all three parameters, we identified 15 modeling studies published after 2020. More than half (n=8, 53%) cited previous modeling studies rather than primary evidence. Citation chain analysis revealed that tracing these citations back to their original sources led to observational studies from the 1990s or early 2000s, but citations of modeling studies made evidence appear relatively recent.
CONCLUSIONS: Current parameter sourcing practices fall substantially short of guideline standards, with modeling studies citing other modeling studies. Our analysis reveals a significant gap between what guidelines recommend and what is practically achievable under traditional workflows. Generative AI tools, with their ability to rapidly synthesize large bodies of evidence, offer a promising pathway to bridge this gap. By automating the labor-intensive process of evidence review, GenAI could enable modelers to move toward the guideline-recommended standard of considering "the full array of available information," ultimately improving the quality of results.