INDIVIDUAL PICO REVIEW AND HIERARCHY IN SYSTEMATIC LITERATURE REVIEWS: WHY GUIDELINE ADHERENCE MATTERS AND HOW REAL-SLRS REDUCE REWORK
Author(s)
Rhiannon Campden, PhD1, Rozee Liu, MSc1, Arup Pramanik, MSc, MD, MBA2, Anna Forsythe, MBA, MSc, PharmD1;
1Oncoscope-AI, Miami, FL, USA, 2Boehringer Ingelheim, Sharon, MA, USA
1Oncoscope-AI, Miami, FL, USA, 2Boehringer Ingelheim, Sharon, MA, USA
OBJECTIVES: Systematic literature review (SLR) guidelines specify records be evaluated sequentially by Population, Intervention/Comparator, Outcomes, and Study design (PI/COS). In practice, human reviewers often use pattern recognition and keyword-based judgments. Available artificial intelligence (AI) tools that support SLRs are trained on human review and therefore replicate these shortcuts. We examined non-hierarchical PICO evaluation to demonstrate how a Real-time AI-assisted Living Systematic Literature Review (REAL-SLR) approach that preserves individual PICO decisions can reduce errors, rework, and time loss when review criteria change.
METHODS: We compared traditional SLR workflows (manual human review and AI-assisted screening) with a REAL-SLR approach that evaluates and stores acceptance or rejection decisions for each PICO element. A prostate cancer SLR was used as an illustrative case. The initial SLR excluded studies involving active surveillance based on predefined intervention criteria. Following a scope change, the intervention criteria were expanded to include active surveillance.
RESULTS: Under traditional workflows, modification of the intervention criteria required re-review of all 9,302 records regardless of rejection reason, resulting in 75 (150) hours of manual (dual) review time or >35 hours using AI-assisted review1, plus ~20 hours to update PRISMA diagrams and SLR documentation. In comparison, since the REAL-SLR retained rejection reasons by individual PICO element, when the intervention criterion was modified only 109 records rejected for the intervention criterion were re-evaluated. Automated updates to the evidence set, PRISMA flow diagram, and reports were generated, and even with 100% human quality control, the total effort required was 3 hours.
CONCLUSIONS: Failure to adhere to hierarchical PICO review leads to inefficiencies, errors, and repeated effort following PICO criteria changes. REAL-SLR systems that preserve independent PICO-level decisions enable rapid, targeted updates while maintaining methodological rigor. The REAL-SLR approach reduces time, cost, reviewer burden, and supports more agile evidence synthesis for HTA and market access decision making.
METHODS: We compared traditional SLR workflows (manual human review and AI-assisted screening) with a REAL-SLR approach that evaluates and stores acceptance or rejection decisions for each PICO element. A prostate cancer SLR was used as an illustrative case. The initial SLR excluded studies involving active surveillance based on predefined intervention criteria. Following a scope change, the intervention criteria were expanded to include active surveillance.
RESULTS: Under traditional workflows, modification of the intervention criteria required re-review of all 9,302 records regardless of rejection reason, resulting in 75 (150) hours of manual (dual) review time or >35 hours using AI-assisted review1, plus ~20 hours to update PRISMA diagrams and SLR documentation. In comparison, since the REAL-SLR retained rejection reasons by individual PICO element, when the intervention criterion was modified only 109 records rejected for the intervention criterion were re-evaluated. Automated updates to the evidence set, PRISMA flow diagram, and reports were generated, and even with 100% human quality control, the total effort required was 3 hours.
CONCLUSIONS: Failure to adhere to hierarchical PICO review leads to inefficiencies, errors, and repeated effort following PICO criteria changes. REAL-SLR systems that preserve independent PICO-level decisions enable rapid, targeted updates while maintaining methodological rigor. The REAL-SLR approach reduces time, cost, reviewer burden, and supports more agile evidence synthesis for HTA and market access decision making.
Conference/Value in Health Info
2026-05, ISPOR 2026, Philadelphia, PA, USA
Value in Health, Volume 29, Issue S6
Code
HTA82
Topic
Health Technology Assessment
Topic Subcategory
Systems & Structure
Disease
SDC: Oncology