Presentation (CTI)

OBJECTIVES: AI agents represent a new avenue to explore in HEOR, offering unprecedented opportunities to enhance human expertise and streamline complex workflows. HEOR analysts often face challenges in identifying appropriate methodological guidance. This research aimed to develop and evaluate an AI-powered command-line interface (CLI) agent that recommends relevant methodological guidance documents based on the analysts’ project requirements, ensuring methodological rigor and improving efficiency in HEOR evidence generation workflows.
METHODS: The agent was developed using the LangGraph software development kit in Python with specialised tools to match user queries with appropriate NICE Technical Support Documents (TSD). The agent features natural language processing capabilities to understand project descriptions, directed knowledge of TSD content through pre-specified tools, and integrated download functionality. Performance was evaluated through: accuracy testing of TSD recommendations against human selection; time saving analysis measuring efficiency gains; and qualitative feedback to assess usability and workflow integration.
RESULTS: The AI agent demonstrated high accuracy, comparable to human selection, in recommending relevant TSDs across diverse HEOR activities, significantly reducing time spent searching for appropriate guidance. The agent successfully provided contextual explanation of technical content, helping users apply described methods. User testing yielded positive qualitative feedback, with consensus that the CLI was easy to operate and easily integrated into existing workflows. Users suggested developing a graphical chat-bot interface could enhance experience for less technical users.
CONCLUSIONS: The AI-powered agent successfully streamlined the synthesis of methodological guidance. While NICE TSDs served as a pilot use case, this approach demonstrates potential for developing multi-agent systems for similar use-cases. By reducing the burden of navigating technical documentation, analysts can focus on the analytical and interpretive aspects of their work. Future research and development activities should focus on specialised tools for use with AI-agents and further implementation of human-in-the-loop approaches to ensure expert judgment remains central while maximizing efficiency.