OBJECTIVES:

Literature analysis could benefit from machine-learning (ML) methods that parse medical text to extract reported results. We demonstrate this by automating the creation of comparative effectiveness Evidence-Gap Maps.

METHODS:

We ran an ML algorithm, using Text Classifiers and Named Entity Recognizers, against a corpus of PubMed abstracts. The algorithm identified the study type (e.g., Randomized Control Trial), and parsed diseases and conditions, interventions, group-sizes and fractional outcomes from the abstract text (e.g., the phrase, "RA clinical remission was 6 of 8 for infliximab" yields {condition: rheumatoid arthritis; intervention: infliximab; outcome: clinical remission; outcome-count: 6; group-size: 8}). Text-spans identifying conditions, interventions, and outcomes were normalized across documents using clustering and ontology alignment.

To construct an Evidence-Gap Map for a given disease or condition, the N most common interventions and outcomes are determined from the extracted data (N=10). The counts of abstracts pairing each intervention and outcome are then visualized as a bubble map using standard plotting software.

To validate, we generated maps for a random sample of conditions with >=100 PubMed abstracts. Spurious normalizations (e.g., "primary outcome") were manually identified and removed. Condition-intervention-outcome triplets were then manually converted into PubMed search queries (e.g., "rheumatoid arthritis" infliximab "clinical remission"), and the results were tallied for gap and non-gap triplets.

RESULTS:

The initial sample showed an average of 0.34 and 4.68 papers for gap and non-gap triplets. Removal of spurious normalizations brought the averages to 0 and 4.7. Search results for those triplets returned an average of 3.2 and 25 results.

CONCLUSIONS:

We demonstrated that machine learning methods can be used to create automated Evidence-Gap Maps which could potentially help identify comparative effectiveness evidence gaps in medical literature.