AI-Assisted Evidence Generation and HTA: Why Transparency and Reproducibility Will Define Trust
Mallik Greene, PhD, DBA, BPharm, Abbott, Boston, Massachusetts, USA
Introduction
When a health technology assessment (HTA) agency receives a dossier in which artificial intelligence (AI) assisted with literature screening, evidence synthesis, statistical programming, and economic modeling, what documentation should the assessors expect before trusting the evidence?
This question is already relevant across health economics and outcomes research (HEOR) and real-world evidence (RWE). AI earned first place in ISPOR’s 2026-2027 Top 10 HEOR Trends, rising from the third position in the 2024-2025 list. Automation of evidence synthesis was anticipated more than a decade ago; generative AI has brought that ambition closer to practical use at scale. Reports from the ISPOR Working Group on Generative AI describe applications in systematic reviews, RWE analysis, economic modeling, and dossier development, and assess opportunities, challenges, and policy considerations specific to HTA. Evaluations show potential efficiency gains for evidence synthesis but also meaningful, variable errors, reinforcing the need for human verification. This article focuses on generative AI, particularly large language models (LLMs), whose probabilistic outputs raise questions about reproducibility and traceability distinct from traditional machine learning.
Healthcare decision makers depend on evidence that is transparent, reproducible, and scientifically defensible. As AI enters evidence-generation workflows, stakeholders are asking: How was it used? What human oversight was applied? Can the workflow be reproduced and validated? Who remains accountable for scientific integrity?
The central challenge is no longer whether AI will be used in evidence generation, but whether stakeholders will trust the resulting evidence.
Some evaluated tools already perform well in bounded tasks such as screening or structured extraction, but performance remains task- and implementation-dependent. Without safeguards, AI-assisted workflows can introduce hallucinations, undocumented decisions, automation bias, inconsistent outputs, and limited auditability, all of which must be weighed against the potential benefits of AI adoption in HEOR. Productivity alone will not establish the scientific credibility that decision makers need.
Human scientific oversight remains central. Qualified experts must verify source fidelity, critically appraise outputs, resolve discrepancies, and remain accountable for methods, interpretation, and scientific validity.
Where AI Is Already Entering Evidence Generation
AI-assisted technologies are being integrated across the evidence life cycle (Figure 1). In systematic reviews, they can support searching, screening, data extraction, summarization, and evidence-table development. In RWE study planning and analysis, generative AI is being explored to process structured and unstructured clinical data and support code generation, exploratory analysis, and interpretation. It can also assist with protocols, statistical programming, economic models, dossiers, and scientific writing.
Figure 1. AI Integration Across the HEOR and RWE Evidence Life Cycle
In digital health and patient-centered evidence generation, AI-enabled analytics are being integrated with wearable and biosensor data for continuous monitoring, pattern detection, and prediction, generating digital endpoints that may feed RWE studies and HTA submissions.
Most organizations are not deploying fully autonomous evidence-generation systems. AI is more commonly implemented as a copilot, with researchers, statisticians, and HEOR scientists retaining responsibility for methods, interpretation, and scientific review. Even in hybrid workflows, AI outputs can influence downstream decisions.
The Emerging Scientific Credibility Gap
Reproducibility is one of the clearest challenges. LLM outputs can change with the model version, prompt wording, configuration, and stochastic settings. HEOR guidance, including the ELEVATE-GenAI framework, therefore recommends documenting model characteristics, prompts, data sources, and validation procedures. Without this information, an analysis may be difficult to reconstruct or independently verify.
Transparency presents a related challenge. A 2025 review of 159 clinical LLM studies found that exact prompt wording was reported in 49%, query dates in 51%, and stochasticity management in only 15%. Within HEOR, the editors introducing Value in Health’s 2025 generative AI section observed that none of the articles to which ELEVATE-GenAI could have been applied would have fully met its criteria. Missing documentation limits reconstruction and obscures the boundary between automated output and human judgment.
Hallucinations and factual inaccuracies remain important concerns, particularly in evidence synthesis. A recent systematic review found substantial and highly variable error rates across searching, screening, extraction, and risk-of-bias assessment. Automation bias can compound these risks when users accept fluent outputs without adequate source verification. These risks are summarized in Figure 2.
Figure 2. Scientific Risks and Principal Safeguards for AI-Assisted Evidence
Policy and governance expectations are already evolving. The European Union AI Act establishes risk-based obligations. The US Food and Drug Administration’s 2025 draft guidance proposes a context-of-use, risk-based credibility framework for AI models supporting regulatory decisions about drugs and biological products. The US National Institute of Standards and Technology (NIST) Generative AI Profile similarly emphasizes life cycle risk management.
HTA agencies have also begun to state their expectations. A 2024 position statement from the National Institute for Health and Care Excellence in the United Kingdom asks submitters to declare and justify AI use, follow existing methods guidance, engage early when planning its use, and retain human oversight and accountability; Canada’s Drug Agency issued a closely aligned statement in 2025. The 2025 HTAi Global Policy Forum discussed the themes of trust, human agency, and risk-based approaches when prioritizing next steps for the integration of GenAI into HTA. Subsequent commentary has focused on translating those principles into validation and accountability in HTA practice.
What Standards Are Needed?
HEOR is not starting from zero. The ISPOR Working Group on Generative AI has published a taxonomy, an HTA-focused report, and ELEVATE-GenAI, which specifies what LLM-assisted studies should report about models, prompts, accuracy, and reproducibility. The remaining gap lies less in what to disclose than in how assessors will judge fitness for a decision, which validation benchmarks and shared test sets each task requires, and how organizations govern AI use across projects.
From the evidence-generator and submitter perspective, 4 principles frame the controls needed for decision-ready evidence (Figure 3).
Figure 3. Four Pillars of Trustworthy AI-Assisted Evidence Generation
First, transparency requires disclosure of where and how AI was used. Documentation should identify the purpose, model and version, prompts or prompt logic, workflow steps, data sources, validation procedures, and known limitations.
Second, reproducibility requires audit trails, version control, and workflow records that allow analyses to be reviewed, replicated, and validated. Criteria and performance thresholds should be defined before use whenever AI influences evidence selection, analytic code, or model inputs.
Third, governance requires standard operating procedures, risk classification, tool validation, monitoring, escalation pathways, and clear accountability. Controls should be proportionate to the task, uncertainty, and potential consequence of error.
Finally, human scientific oversight remains central. Qualified experts must verify source fidelity, critically appraise outputs, resolve discrepancies, and remain accountable for methods, interpretation, and scientific validity.
HTA agencies and payers need evidence-lineage records, reproducibility safeguards, and transparent validation before accepting AI-assisted evidence.
Lessons Learned
Recent evidence suggests a practical lesson: Efficiency gains are credible only when the workflow can be reconstructed. Generative AI can assist evidence synthesis, and some tools perform well in bounded tasks, but performance varies and material errors remain possible.
An AI-assisted literature-screening workflow may appear faster than traditional methods yet still be unsuitable for decision making if the model version, prompts, source set, inclusion criteria, and validation results are not preserved. Without that record, teams may be unable to explain or reproduce why individual studies were included, excluded, or summarized in a particular way. The same principle applies to code generation, RWE analysis, economic modeling, and dossier development.
A tool used only to improve wording does not require the same validation as a tool used to select evidence, generate analytic code, or influence model inputs. This distinction is central to proportionate, risk-based governance.
HTA agencies and payers need evidence-lineage records, reproducibility safeguards, and transparent validation before accepting AI-assisted evidence. HEOR and RWE teams need documented workflows, version control, and oversight calibrated to the task and consequences of error. Journals and regulators need to set clear disclosure expectations. Patient organizations may also expect transparency about whether AI contributed to evidence supporting treatment, access, or reimbursement decisions.
ISPOR has already shaped this agenda through the Working Group on Generative AI, ELEVATE-GenAI, and its AI resources and policy work. The next opportunity is to translate that foundation into HTA-side acceptance criteria; task-specific validation benchmarks and shared test cases; proportionate human-review requirements; and auditability standards across systematic reviews, RWE analyses, economic models, and HTA submissions. Consensus on decision-ready documentation would help organizations implement these principles consistently in practice.
Future progress will depend not only on what AI systems can do, but on whether the HEOR community can make their use transparent, reproducible, accountable, and worthy of trust.
Author disclosure and disclaimer: Mallik Greene is an employee of Abbott. This article was prepared independently in the author’s personal capacity. The views expressed are those of the author and do not necessarily reflect the views or positions of Abbott.
AI Use Disclosure Statement: OpenAI ChatGPT (GPT-5.6 Pro) and Anthropic Claude were used to support editorial refinement, manuscript review, drafting of revision text, and figure-content and layout development. ChatGPT (GPT-6 Astra Pro) also supported this copyediting round. Python 3 with Matplotlib was used to render the figures; Pillow was used to replace the Figure 1 title band. All AI-assisted output was reviewed, verified against the cited sources, and edited by the author, who retains full responsibility for the scientific integrity, accuracy, and interpretation of the article.
