Conference proceeding
Knowledge-Guided Efficient Representation Learning for Biomedical Domain
KDD '21: Proceedings of the 27th Acm SIGKDD Conference on Knowledge Discovery & Data Mining, pp.3077-3085
01/01/2021
DOI: 10.1145/3447548.3467118
Abstract
Pre-trained concept representations are essential to many biomedical text mining and natural language processing tasks. As such, various representation learning approaches have been proposed in the literature. More recently, contextualized embedding approaches (i.e., BERT based models) that capture the implicit semantics of concepts at a granular level have significantly outperformed the conventional word embedding approaches (i.e., Word2Vec/GLoVE based models). Despite significant accuracy gains achieved, these approaches are often computationally expensive and memory inefficient. To address this issue, we propose a new representation learning approach that efficiently adapts the concept representations to the newly available data. Specifically, the proposed approach develops a knowledge-guided continual learning strategy wherein the accurate/stable context-information present in human-curated knowledge-bases is exploited to continually identify and retrain the representations of those concepts whose corpus-based context evolved coherently over time. Different from previous studies that mainly leverage the curated knowledge to improve the accuracy of embedding models, the proposed research explores the usefulness of semantic knowledge from the perspective of accelerating the training efficiency of embedding models. Comprehensive experiments under various efficiency constraints demonstrate that the proposed approach significantly improves the computational performance of biomedical word embedding models.
Details
- Title: Subtitle
- Knowledge-Guided Efficient Representation Learning for Biomedical Domain
- Creators
- Kishlay Jha - University of VirginiaGuangxu Xun - University of VirginiaNan Du - nLIGHT (United States)Aidong Zhang - University of Virginia
- Resource Type
- Conference proceeding
- Publication Details
- KDD '21: Proceedings of the 27th Acm SIGKDD Conference on Knowledge Discovery & Data Mining, pp.3077-3085
- DOI
- 10.1145/3447548.3467118
- Publisher
- Association of Computing Machinery
- Number of pages
- 9
- Grant note
- IIS-2008208; 1934600; 1938167; 1955151 / US National Science Foundation; National Science Foundation (NSF)
- Language
- English
- Date published
- 01/01/2021
- Academic Unit
- Electrical and Computer Engineering
- Record Identifier
- 9984295027502771
Metrics
47 Record Views