Extracting Biomedical Factual Knowledge Using Pretrained Language Model and Electronic Health Record Context

by   Zonghai Yao, et al.

Language Models (LMs) have performed well on biomedical natural language processing applications. In this study, we conducted some experiments to use prompt methods to extract knowledge from LMs as new knowledge Bases (LMs as KBs). However, prompting can only be used as a low bound for knowledge extraction, and perform particularly poorly on biomedical domain KBs. In order to make LMs as KBs more in line with the actual application scenarios of the biomedical domain, we specifically add EHR notes as context to the prompt to improve the low bound in the biomedical domain. We design and validate a series of experiments for our Dynamic-Context-BioLAMA task. Our experiments show that the knowledge possessed by those language models can distinguish the correct knowledge from the noise knowledge in the EHR notes, and such distinguishing ability can also be used as a new metric to evaluate the amount of knowledge possessed by the model.


Can Language Models be Biomedical Knowledge Bases?

Pre-trained language models (LMs) have become ubiquitous in solving vari...

Context Variance Evaluation of Pretrained Language Models for Prompt-based Biomedical Knowledge Probing

Pretrained language models (PLMs) have motivated research on what kinds ...

Empowering Language Model with Guided Knowledge Fusion for Biomedical Document Re-ranking

Pre-trained language models (PLMs) have proven to be effective for docum...

Large Language Models, scientific knowledge and factuality: A systematic analysis in antibiotic discovery

Inferring over and extracting information from Large Language Models (LL...

Exploring the In-context Learning Ability of Large Language Model for Biomedical Concept Linking

The biomedical field relies heavily on concept linking in various areas ...

Large Language Models with Controllable Working Memory

Large language models (LLMs) have led to a series of breakthroughs in na...

Hierarchical Pretraining for Biomedical Term Embeddings

Electronic health records (EHR) contain narrative notes that provide ext...

Please sign up or login with your details

Forgot password? Click here to reset