Hierarchical Pretraining on Multimodal Electronic Health Records

Xiaochen Wang, Junyu Luo, Jiaqi Wang, Ziyi Yin, Suhan Cui +3 more
10/11/2023
cs.AI

No official code found. Best implementation: XiaochenWang-PSU/MedHMP (21 stars, last commit 2024-04-10).

Abstract

Pretraining has proven to be a powerful technique in natural language processing (NLP), exhibiting remarkable success in various NLP downstream tasks. However, in the medical domain, existing pretrained models on electronic health records (EHR) fail to capture the hierarchical nature of EHR data, limiting their generalization capability across diverse downstream tasks using a single pretrained model. To tackle this challenge, this paper introduces a novel, general, and unified pretraining framework called MEDHMP, specifically designed for hierarchically multimodal EHR data. The effectiveness of the proposed MEDHMP is demonstrated through experimental results on eight downstream tasks spanning three levels. Comparisons against eighteen baselines further highlight the efficacy of our approach.

View on arXivView PDF

Code Implementations(1)

XiaochenWang-PSU/MedHMPCommunity100%

Source codes of the paper "Hierarchical Pretraining on Multimodal Electronic Health Records".

214PythonOct 17, 20232 years ago

Similar Papers

Harnessing the Power of BERT in the Turkish Clinical Domain: Pretraining Approaches for Limited Data Scenarios

85% match2023

DrBERT: A Robust Pre-trained Model in French for Biomedical and Clinical domains

84% match1 repo222023

TCM-GPT: Efficient Pre-training of Large Language Models for Domain Adaptation in Traditional Chinese Medicine

83% match2023

PLM-ICD: Automatic ICD Coding with Pretrained Language Models

83% match1 repo802022

Enhancing Medical Specialty Assignment to Patients using NLP Techniques

82% match2023

GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learning

82% match2 repos242022

Cite this paper

@article{wang2023hierarchical,
  title  = {Hierarchical Pretraining on Multimodal Electronic Health Records},
  author = {Xiaochen Wang and Junyu Luo and Jiaqi Wang and Ziyi Yin and Suhan Cui and Yuan Zhong and Yaqing Wang and Fenglong Ma},
  year   = {2023},
  eprint = {2310.07871},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.07871v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.07871.svg)](https://codeofpaper.com/paper/2310.07871)

Discussion