Developing a Large Language Model-Based Simulation for P&T Committees: Methodology and Inputs for Realistic Dynamics
Author(s)
Leanna Baker Williams, PharmD1, Amarri Trueheart, PharmD1, Nathan Schurr, Ph.D.2, Casey Dobie, PharmD1, Malia Gill, MS1.
1Cencora, Conshohocken, PA, USA, 2AGI Technology Partners, Poway, CA, USA.
1Cencora, Conshohocken, PA, USA, 2AGI Technology Partners, Poway, CA, USA.
Presentation Documents
OBJECTIVES: Pharmacy & Therapeutics (P&T) committees are vital for the management of drug formularies. Decisions vary between organizations, making the formulary decision-making process unpredictable. Our objective was to understand the potential of using a large language model (LLM) to simulate P&T discussions.
METHODS: Cencora’s internal generative artificial intelligence (AI) platform created a P&T meeting simulation. It allows custom personas and leverages a privately deployed instance of OpenAI’s GPT-4o, a GPT-4 variant with quicker responses and more efficient output. For the P&T simulation, committee members varied in subject matter expertise, personal sensitivities, attitudes toward the pharmaceutical industry, and professional backgrounds. Each member was assigned a committee role, and an LLM translated these characteristics into virtual personas. The model facilitated discussion and members deliberated, cast their votes, and created a meeting summary with final recommendation based on majority consensus.
RESULTS: The LLM successfully followed instructions and created realistic discussions. There were 0 accounts where members spoke outside their areas of expertise. Members maintained personality attributes and offered perspectives matching designated traits. The committee chair proficiently summarized the meeting and concluded with recommendations aligned with the consensus. Of note, virtual personas had difficulty with productive discourse during disagreements. Limitations include issues inherent to generative AI platforms, including knowledge being limited to the programmers’ input. Contributions were sometimes limited by subject matter expertise, and clinical subtleties were not consistently considered by all members.
CONCLUSIONS: This study illustrates potential advantages of using LLM to simulate P&T committee discussions. Members consistently adhered to personality attributes fostering diversity and reflecting realistic committee dynamics. Future work will focus on improving the model’s coherence and expanding its ability to handle more complex documents.As stated herein, the evaluation was supported by Cencora’s proprietary AI platform, which was used in accordance with Cencora’s AI policies, and reviewed by a human.
METHODS: Cencora’s internal generative artificial intelligence (AI) platform created a P&T meeting simulation. It allows custom personas and leverages a privately deployed instance of OpenAI’s GPT-4o, a GPT-4 variant with quicker responses and more efficient output. For the P&T simulation, committee members varied in subject matter expertise, personal sensitivities, attitudes toward the pharmaceutical industry, and professional backgrounds. Each member was assigned a committee role, and an LLM translated these characteristics into virtual personas. The model facilitated discussion and members deliberated, cast their votes, and created a meeting summary with final recommendation based on majority consensus.
RESULTS: The LLM successfully followed instructions and created realistic discussions. There were 0 accounts where members spoke outside their areas of expertise. Members maintained personality attributes and offered perspectives matching designated traits. The committee chair proficiently summarized the meeting and concluded with recommendations aligned with the consensus. Of note, virtual personas had difficulty with productive discourse during disagreements. Limitations include issues inherent to generative AI platforms, including knowledge being limited to the programmers’ input. Contributions were sometimes limited by subject matter expertise, and clinical subtleties were not consistently considered by all members.
CONCLUSIONS: This study illustrates potential advantages of using LLM to simulate P&T committee discussions. Members consistently adhered to personality attributes fostering diversity and reflecting realistic committee dynamics. Future work will focus on improving the model’s coherence and expanding its ability to handle more complex documents.As stated herein, the evaluation was supported by Cencora’s proprietary AI platform, which was used in accordance with Cencora’s AI policies, and reviewed by a human.
Conference/Value in Health Info
2025-05, ISPOR 2025, Montréal, Quebec, CA
Value in Health, Volume 28, Issue S1
Code
MSR143
Topic
Methodological & Statistical Research
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics
Disease
SDC: Diabetes/Endocrine/Metabolic Disorders (including obesity)