Preprint
Adaptive Perturbation Selection for Contrastive Audio Decoding
ArXiv.org
arXiv
06/30/2026
DOI: 10.48550/arxiv.2607.00247
Abstract
Large audio-language models (LALMs) frequently hallucinate by overriding acoustic evidence with language priors. While contrastive decoding (CD) offers training-free mitigation, existing methods rely on blunt perturbations like masking or noise, leaving structured audio transformations unexplored. We explore this design space by evaluating a diverse library of targeted audio perturbations and adaptively selecting the optimal negative branch for each task and example. First, we improve upon earlier prompt engineering by showing that a simple binary yes/no constraint reduces the model's tendency to falsely confirm absent audio features. Second, evaluating our library across temporal, spectral, frequency, and amplitude domains reveals that optimal transformations are highly task-dependent; for instance, reversing the audio array disrupts temporal coherence, raising accuracy on the temporal order task from 74.7% to 81.4%. Finally, we trained a light-weight perturbation selector on model hidden states to dynamically route negative branches, yielding an additional +4.3% gain on the existence task.
Details
- Title: Subtitle
- Adaptive Perturbation Selection for Contrastive Audio Decoding
- Creators
- Aaron Isidore GraceZhouyuan HuoWeiran Wang
- Resource Type
- Preprint
- Publication Details
- ArXiv.org
- DOI
- 10.48550/arxiv.2607.00247
- ISSN
- 2331-8422
- Publisher
- arXiv
- Language
- English
- Date posted
- 06/30/2026
- Academic Unit
- Computer Science
- Record Identifier
- 9985179093902771
Metrics
1 Record Views