Presentation (CTI)

OBJECTIVES: Evidence synthesis typically requires searching multiple databases to ensure comprehensive data collection, making deduplication an essential stage. We prospectively evaluated the performance of the rule-based algorithm used in the SuperDeduper deduplication module within Laser AI, extending our earlier retrospective assessment, and characterized performance by confidence level, including the proportion of records routed for human review.
METHODS: Three systematic reviews of various sizes (3,645 to 31,740 records) and topics were selected and manually deduplicated independently by two researchers to create benchmark reference sets. The deduplication algorithm groups potential duplicates into four confidence levels. Performance was assessed using false positives, false negatives, sensitivity, specificity, and accuracy, overall and stratified by confidence level.
RESULTS: The benchmark dataset included 40,574 records. Across the three benchmark sets, average accuracy was 99.4%, average sensitivity was 98.2%, and average specificity exceeded 99.9%. In addition, SuperDeduper identified 4 duplicates that were missed in manual deduplication and incorrectly flagged 12 records as duplicates (false positives). All false positives were assigned to the low-confidence level and therefore were not automatically removed, instead being routed for human review. On average, 6.64% of records were assigned to the low-confidence level requiring manual verification.
CONCLUSIONS: The SuperDeduper module supports a human-in-the-loop workflow by assigning confidence levels to potential duplicate groups, providing users with interpretable and actionable outputs. The results confirm that SuperDeduper is an effective and safe method for removing duplicates and may accelerate the review process by reducing human workload in duplicate verification. The generated benchmark dataset (to be published) offers a valuable resource for further validation of deduplication tools.