Presentation (CTI)

OBJECTIVES: Large language models (LLMs) are increasingly being incorporated into HEOR workflows for model development, critique, implementation, and education. However, prompt engineering remains an emerging discipline, and little guidance exists regarding which prompt components are required for successful health economic model implementation. Many HEOR specifications rely on conventions that experienced modellers infer automatically but rarely document explicitly. Prompt quality may influence implementation accuracy, efficiency, token usage, and the balance of human versus AI effort.
METHODS: We attempted to implement a synthetic oncology partitioned survival model (PSM) using an LLM. During iterative refinement, prompt modifications were documented and categorized according to their impact on implementation success. Dependencies were grouped into HEOR-specific and governance-related domains. The final implementation was evaluated against predefined benchmarks using a ±1% pass/fail criterion.
RESULTS: Fourteen prompt dependencies were identified and grouped into ten domains. HEOR-specific domains included model structure, survival parameterization, state derivation, discounting conventions, cost allocation, and utility calculations. Governance-related domains included input mutability, validation logic, transparency requirements, and benchmarking standards. Several directly altered model outputs while others affected reproducibility, validation, or auditability. Explicit inclusion of these domains eliminated implementation discrepancies and enabled benchmark agreement within ±1%.
CONCLUSIONS: Effective AI-assisted health economic modeling requires more than model specification alone. Prompt quality influenced implementation accuracy, reproducibility, and validation burden. We identified a set of domains that humans infer automatically but AI systems may not. These domains may form the basis of future HEOR prompt-engineering guidance. Future work should quantify their impact on implementation accuracy, token usage, and modeller effort across multiple platforms and model types.