Conference proceeding
LifelongSkill: Toward Modality-Varying Lifelong Learning with Latent Knowledge Hypergraph
Proceedings (IEEE International Conference on Data Mining), pp.1105-1114
11/12/2025
DOI: 10.1109/ICDM65498.2025.00119
Abstract
Human intelligence can continuously and adaptively build multimodal cognition from a series of diverse modalities in the external world. Modality-varying Continual Learning (MVCL) aims to imitate such human intelligence, which trains models on a stream of non-stationary and modality-fluctuating data distributions while sequentially transferring and protecting past knowledge. When an MVCL learner cannot anticipate the complexity of future new modalities and inter-modal interactions, the challenge of dealing with knowledge saturation (KS) with satisfactory parameter efficiency (PE) increases. Existing works focused mainly on overcoming the forgetting of past knowledge but overlooked the critical tradeoff between KS and PEC To address this gap, we propose a novel continual learning frame-work, namely LifelongSkill, that explicitly optimizes this tradeoff. Our key idea is to capture the interpretable inter-task diversity underlying the task stream, and then use this information to guide the parameter-efficient knowledge transfer and necessary network expansion. Specifically, we learn a Latent Knowledge Hypergraph (LKGraph), comprising a variety of semantically-distinct functional capabilities (namely skills) learned from tasks, to represent task diversity through skill co-occurrences. Then, we propose a Skill-wise Node Decoder (SND) to facilitate parameter-efficient network expansion and knowledge transfer guided by LKGraph. Experiment results demonstrate the proposed approach achieves the best tradeoffs between performance and parameter efficiency compared with baselines.
Details
- Title: Subtitle
- LifelongSkill: Toward Modality-Varying Lifelong Learning with Latent Knowledge Hypergraph
- Creators
- Jiayi Chen - University of VirginiaKishlay Jha - University of IowaAidong Zhang - University of Virginia
- Resource Type
- Conference proceeding
- Publication Details
- Proceedings (IEEE International Conference on Data Mining), pp.1105-1114
- DOI
- 10.1109/ICDM65498.2025.00119
- eISSN
- 2374-8486
- Publisher
- IEEE
- Grant note
- U.S. National Science Foundation (10.13039/100000001)
- Language
- English
- Date published
- 11/12/2025
- Academic Unit
- Electrical and Computer Engineering
- Record Identifier
- 9985141959402771
Metrics
1 Record Views