Presentation (CTI)

OBJECTIVES: Large language models enable new applications of artificial intelligence (AI) in healthcare; however, they require domain adaptation and fine-tuning to meet the quality, tone, and domain-specific medical writing tasks. The objective of this proof of concept was to validate a secure, cloud-based framework for fine-tuning of large language models to support writing-intensive HEOR workflows
METHODS: A modular framework was implemented as a proof of concept to enable domain adaptation and fine-tuning of large language models within a secure, cloud-based environment using AWS SageMaker AI to support HEOR workflows. Phase 1: Domain-specific source documents (e.g., published literature, HTA guidelines, dossiers, and study reports) were collected, normalized, chunked, tokenized, and securely stored with encryption. The curated data were then partitioned into training 80% and testing 20% datasets. Phase 2: Fine-tuning was performed on an open-source large language model (LLaMA-2-7B) using multiple adaptation strategies, including QLoRA, LoRA, prefix tuning, adapter tuning. Phase 3: Outputs generated on the testing dataset were assessed for writing quality, tone, style adaptation, and cost considerations, followed by subject matter expert validation to ensure alignment with HTA and regulatory
RESULTS: The framework was validated as a proof of concept using medical writing prompts for dossier, report, and protocol development. Training cost increased with data size across fine-tuning approaches. QLoRA achieved greater cost efficiency but comparatively lower output refinement, whereas LoRA produced higher-quality outputs with improved tonal consistency, clearer structure, and stronger adaptation to medical writing style. Fine-tuned models were generated and deployed securely, supporting the feasibility of the framework for HEOR evidence-generation workflows
CONCLUSIONS: This study demonstrates the feasibility of a secure, cloud-based framework for fine-tuning large language models and presents a novel application of such a framework within HEOR workflows. Future work will focus on enterprise-scale deployment and the integration of domain-adapted models into multi-agent HEOR workflows