Presentation (CTI)

OBJECTIVES: Recent applications of large language models (LLMs) to patient-reported outcome measures (PROMs) enable adaptive, open-ended patient interactions but introduce critical content validity concerns: unconstrained generation risks semantic drift and hallucination, while the absence of structured cognitive debriefing compromises construct fidelity. This research proposes agentic PROs (aPROs), a framework addressing these limitations through knowledge graph-constrained generation that reduces hallucination via explicit concept anchoring, and human-in-the-loop governance that preserves patient-centricity and alignment with established validation standards.
METHODS: The aPRO framework comprises three components. First, concept-anchored generation encodes patient-derived conceptual models as directed knowledge graphs, with nodes representing symptoms, impacts, and behaviors, and edges capturing validated relationships from qualitative sources. Item generation employs retrieval-augmented generation constrained to graph-traversable paths, ensuring traceability from candidate items to source concepts. Second, multi-agent cognitive debriefing validates content through a Persona Agent instantiating diverse respondent profiles across literacy levels, disease severity, and demographics; a Debriefing Agent executing protocols assessing comprehension, response process alignment, and construct coverage; and a context-aware Validator Agent trained on PFDD guidelines and COSMIN criteria. Third, human-in-the-loop governance embeds patient and expert review at concept model approval, item evaluation, and final selection.
RESULTS: This paper presents the aPRO architecture, knowledge graph schema, agent protocols, and validation methodology. The framework generates auditable artifacts preserving concept-to-item traceability and debriefing documentation supporting regulatory submissions.
CONCLUSIONS: The aPRO framework addresses fundamental content validity limitations of LLM-augmented PROMs through concept-anchored generation and agentic cognitive debriefing while preserving human oversight central to patient-focused development. Future research will benchmark aPRO-validated items against traditional cognitive debriefing outcomes.