Presentation (CTI)

OBJECTIVES: Health economics and outcomes research (HEOR) requires synthesizing evidence, where identifying relationships between evidence is essential for comprehensive interpretation. Conventional Retrieval-Augmented Generation (RAG) retrieves semantically relevant document passages but has limited ability to leverage relationships between evidence during retrieval. This study compared conventional RAG with Graph Retrieval-Augmented Generation (GraphRAG) to evaluate their impact on evidence retrieval, contextual understanding, and evidence synthesis for HEOR applications
METHODS: A proof-of-concept (PoC) compared two retrieval pipelines: (1) a conventional RAG framework using semantic document retrieval and (2) a GraphRAG framework integrating semantic retrieval with a knowledge graph. Published HEOR literature was indexed, and key entities and their relationships were automatically extracted and linked to construct the knowledge graph. Evidence retrieved by both pipelines was assembled into structured context before response generation. Outputs were independently evaluated by subject matter experts (SMEs) for retrieval completeness, contextual understanding, evidence traceability, and overall quality.
RESULTS: Compared with conventional RAG, GraphRAG more effectively leveraged relationships between evidence, resulting in improved retrieval completeness, contextual understanding, evidence traceability, and the coherence of generated evidence summaries. SMEs agreed that GraphRAG-generated reports were more relevant, accurate, and traceable to the source publications than those generated using conventional RAG. However, SMEs also observed that GraphRAG performance depended on accurate graph construction and relationship traversal, particularly when evidence contained complex relationship structures. These findings suggest that GraphRAG requires careful graph design and expert oversight for complex evidence networks.
CONCLUSIONS: GraphRAG extends conventional RAG by incorporating relationship-aware retrieval, enabling more comprehensive evidence synthesis for HEOR applications. However, its effectiveness depends on accurate graph construction and relationship complexity, highlighting the need for careful implementation and expert oversight.