Attention Is All You Need (to Avoid Spurious Oscillations)

Jinyoung Jeong; Joseph B Choi; Xinlun Cheng; H. S Udaykumar; Sanghun Choi; Stephen S Baek

doi:10.48550/arxiv.2609.13531

Back

Attention Is All You Need (to Avoid Spurious Oscillations)

Preprint

Open access

Attention Is All You Need (to Avoid Spurious Oscillations)

Jinyoung Jeong, Joseph B Choi, Xinlun Cheng, H. S Udaykumar, Sanghun Choi and Stephen S Baek

arXiv

09/11/2026

DOI: 10.48550/arxiv.2609.13531

Files and links (1)

url

https://doi.org/10.48550/arxiv.2609.13531View

Preprint (Author's original) This preprint has not been evaluated by subject experts through peer review. Preprints may undergo extensive changes and/or become peer-reviewed journal articles. Open Access

Abstract

Can attention move a shock across several cells in one update without breaking it? We develop a conservative, fixed grid finite-volume scheme in which a CFL-conditioned attention flux selects upstream information according to the transport required by the current time step. One-dimensional inviscid Burgers transport is used as the central mechanism test: the same learned flux remains reliable in the conventional small-step regime and, with a time step four times larger, preserves sharp shocks while using one stage per update. A standard fifth-order WENO scheme with third-order strong-stability-preserving Runge-Kutta time integration (WENO-5+SSP-RK3) is included alongside controlled Forward Euler comparisons to separate flux selection from time integration. The learned attention shifts upstream with the local transport reach and becomes more selective near shocks; inference-time interventions and retrained ablations show that transport-scale information and state-dependent selection contribute directly to performance. Directional two-dimensional scalar Burgers transport and the one-dimensional shallow-water system then test whether the conservation-scale-selection principle transfers beyond the original scalar setting. The results support attention as a learnable information stencil for conservative large-step shock transport, while identifying finite candidate reach and problem-dependent robustness as the present limits.

Computer Science - Artificial Intelligence

Computer Science - Learning

Physics - Fluid Dynamics

Details

Title: Subtitle: Attention Is All You Need (to Avoid Spurious Oscillations)
Creators: Jinyoung Jeong
Joseph B Choi
Xinlun Cheng
H. S Udaykumar
Sanghun Choi
Stephen S Baek
Resource Type: Preprint
Publication Details: arXiv
DOI: 10.48550/arxiv.2609.13531
ISSN: 2331-8422
Publisher: arXiv
Language: English
Date posted: 09/11/2026
Academic Unit: Engineering Administration; Injury Prevention Research Center; Chemical and Biochemical Engineering; Mechanical Engineering
Record Identifier: 9985230927202771

Metrics

1 Record Views