OpenResearchAlpha

JMIR Cardio · 20 Nov 2026

Comparison of Rule-Based Natural Language Processing and Diagnosis Code Approaches to Detecting Bleeding in the Electronic Health Record: Algorithm Validation Within a Retrospective Cohort Study

Evan W Draperh-index 1, Jan A Anderson, Surbhi Shah, Leilia Brown, Margaret R Hanson, Kristin D Negley, Jennifer M Lutz, Andrea E Anderson, John G O'Meara, Nicholas D Jakowenko, Robert D McBaneh-index 49

0
Citations
no database records one yet
0
Citations in 2026
so far
49
Leading author's h-index
Robert D McBane, in QAI's scholar index
26
References
works it cites, from open citation data

Citations by database

Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.

  • OpenCitations0
  • Crossref0
  • Europe PMC0

Citations per year

The current year is still counting.

  • 2026 so far0

Abstract

Abstract Background Anticoagulant use is prevalent and increasing. High-quality anticoagulation management balances individual thrombotic and bleeding risks to optimize care. An automated algorithm to accurately extract major bleeding events from the electronic health record (EHR) of a large health care system would be extremely useful for tracking outcomes of anticoagulated patients. Objective The study aims to assess the accuracy of a natural language processing (NLP) approach, compared with and in conjunction with a diagnosis code (Dx Code)–based strategy, for detecting major bleeding from the EHR. Methods Patients with at least 1 prescription for apixaban, dabigatran, edoxaban, rivaroxaban, or warfarin between January 1, 2019, and December 31, 2023, were analyzed for major bleeding events. Major bleeding was defined similarly to the International Society on Thrombosis and Haemostasis definition. For the NLP algorithm, clinical notes and radiology reports were screened for mentions of bleeding. MedTagger, an open-source, rule-based NLP tool, was then used on the notes. The NLP results were transformed into a feature set, which was used to create a support vector machine model. Three NLP models were created for central nervous system bleeding, critical site bleeding, and bleeding with a hemoglobin drop or transfusions (other major bleeding). Training and test datasets were created using an active learning approach. The model was trained using 80% of the dataset, while the remaining 20% was used for testing. After model finalization, an independent random sample was used to validate the model. Dx Codes were also used to detect major bleeding, similar to previously published approaches. Five unique algorithms were compared: (1) NLP, (2) Dx Code in the primary position, (3) Dx Code in any position, (4) NLP∩Dx Code in any position, and (5) NLP∪Dx Code in any position. All 5 algorithms were compared with manual chart review. Results The final analysis identified 111,338 patients prescribed outpatient anticoagulation during the study. Manual adjudication was performed for 1189 of the 111,338 (1%) patients. Most of the 5 approaches performed well on the test data, with F 1 -scores ranging from 0.545 to 0.959, except for Dx Code–Primary, with an F 1 -score of 0.353 for critical site bleeding. The F 1 -scores for the validation data ranged from 0.509 to 0.679. The NLP∩Dx Code in any position approach yielded excellent precision of 0.864 (95% CI 0.667‐0.953) but only 0.559 (95% CI 0.395‐0.711) sensitivity. The NLP∪Dx Code in any position approach yielded good sensitivity of 0.824 (95% CI 0.665‐0.917) but only 0.368 (95% CI 0.269‐0.481) precision. Conclusions Electronic tools can screen the EHRs to identify major bleeding outcomes in large patient cohorts within a multisite health care network with high precision but only moderate sensitivity. Additional work is needed to enhance sensitivity while maintaining high precision.

What the data shows

  • No database records a citation yet, as is usual for a paper this new.
  • Robert D McBane has the highest h-index among the authors (49).

Graph

This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.

Citing paper shares an authorAuthor without a profileSize shows citations, or h-index for authors

Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.

Start with

Authors (11)

Each author's standing in QAI's scholar index: the works their ORCID iD is attached to, and for anyone whose ORCID record names a lab OpenResearch follows, their arXiv preprints matched by name and a shared co-author. Names are as printed on the paper.

ProfileMost-cited paper

Cited by 0 papers

Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.

0 shown
No citing papers for this filter.

References (26)

What it cites, from open citation data, with Crossref's record of each.

Sign in to see 6 more references

A free account opens the rest of the citing papers and references on every paper’s page.

Comparison of Rule-Based Natural Language Processing and Diagnosis Code Approaches to Detecting Bleeding in the Electronic Health Record: Algorithm Validation Within a Retrospective Cohort Study - QAI OpenResearch