Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition

by   Inigo Jauregi Unanue, et al.

Background. Previous state-of-the-art systems on Drug Name Recognition (DNR) and Clinical Concept Extraction (CCE) have focused on a combination of text "feature engineering" and conventional machine learning algorithms such as conditional random fields and support vector machines. However, developing good features is inherently heavily time-consuming. Conversely, more modern machine learning approaches such as recurrent neural networks (RNNs) have proved capable of automatically learning effective features from either random assignments or automated word "embeddings". Objectives. (i) To create a highly accurate DNR and CCE system that avoids conventional, time-consuming feature engineering. (ii) To create richer, more specialized word embeddings by using health domain datasets such as MIMIC-III. (iii) To evaluate our systems over three contemporary datasets. Methods. Two deep learning methods, namely the Bidirectional LSTM and the Bidirectional LSTM-CRF, are evaluated. A CRF model is set as the baseline to compare the deep learning systems to a traditional machine learning approach. The same features are used for all the models. Results. We have obtained the best results with the Bidirectional LSTM-CRF model, which has outperformed all previously proposed systems. The specialized embeddings have helped to cover unusual words in DDI-DrugBank and DDI-MedLine, but not in the 2010 i2b2/VA IRB Revision dataset. Conclusion. We present a state-of-the-art system for DNR and CCE. Automated word embeddings has allowed us to avoid costly feature engineering and achieve higher accuracy. Nevertheless, the embeddings need to be retrained over datasets that are adequate for the domain, in order to adequately cover the domain-specific vocabulary.


page 1

page 2

page 3

page 4


GNTeam at 2018 n2c2: Feature-augmented BiLSTM-CRF for drug-related entity recognition in hospital discharge summaries

Monitoring the administration of drugs and adverse drug reactions are ke...

Bidirectional LSTM-CRF for Clinical Concept Extraction

Automated extraction of concepts from patient clinical records is an ess...

TF-IDF vs Word Embeddings for Morbidity Identification in Clinical Notes: An Initial Study

Today, we are seeing an ever-increasing number of clinical notes that co...

Bidirectional Recurrent Neural Networks for Medical Event Detection in Electronic Health Records

Sequence labeling for extraction of medical events and their attributes ...

Corporate IT-support Help-Desk Process Hybrid-Automation Solution with Machine Learning Approach

Comprehensive IT support teams in large scale organizations require more...

A Deep Representation Empowered Distant Supervision Paradigm for Clinical Information Extraction

Objective: To automatically create large labeled training datasets and r...

When Specialization Helps: Using Pooled Contextualized Embeddings to Detect Chemical and Biomedical Entities in Spanish

The recognition of pharmacological substances, compounds and proteins is...

Please sign up or login with your details

Forgot password? Click here to reset