Logo image
Back
Structured Latent Modeling for Supervised Multimodal Information Decomposition
Preprint   Open access

Structured Latent Modeling for Supervised Multimodal Information Decomposition

Wanting Huang, Sanvesh Srivastava and Weiran Wang
arXiv
arXiv
09/28/2026
DOI: 10.48550/arxiv.2609.35502
url
https://doi.org/10.48550/arxiv.2609.35502View
Preprint (Author's original) This preprint has not been evaluated by subject experts through peer review. Preprints may undergo extensive changes and/or become peer-reviewed journal articles. Open Access

Abstract

Multimodal prediction relies on diverse forms of evidence: information repeated across modalities, cues specific to a single source, and complex cross-modal dependencies that emerge only when inputs are considered together. While recent methods promote richer interactions, they lack a principled way to isolate these target-relative contributions within learned continuous representations. We introduce a framework that applies contrastive or masked objectives at intermediate layers, coupled with source-wise invertible normalizing flows and a supervised, low-rank latent variable model. This architecture explicitly factorizes the joint distribution into shared task-relevant variation, modality-specific predictive variation, and task-irrelevant dependence. Drawing connections to prior multimodal learning assumptions, our approach evaluates how modalities independently and jointly contribute to the target. Ultimately, this framework unites intermediate representation learning with structured likelihood-based guidance, offering a practical latent-variable lens for characterizing continuous multimodal interactions. Empirically, we demonstrate the effectiveness of our approach across diverse multimodal benchmarks, showing robust improvements in predictive performance.
 Computer Science - Learning

Details

Metrics

1 Record Views
Logo image