Conference proceeding
Which Way from B to A: The role of embedding geometry in image interpolation for Stable Diffusion
Proceedings of the 1st Conference on Topology, Algebra, and Geometry in Data Science, TAG-DS 2025, Vol.321, pp.1-12
2025
Abstract
It can be shown that Stable Diffusion has a permutation-invariance property with respect to the rows of Contrastive Language-Image Pretraining (CLIP) embedding matrices. This inspired the novel observation that these embeddings can naturally be interpreted as point clouds in a Wasserstein space rather than as matrices in a Euclidean space. This perspective opens up new possibilities for understanding the geometry of embedding space. For example, when interpolating between embeddings of two distinct prompts, we propose reframing the interpolation problem as an optimal transport problem. By solving this optimal transport problem, we compute a shortest path (or geodesic) between embeddings that captures a more natural and geometrically smooth transition through the embedding space. This results in smoother and more coherent intermediate (interpolated) images when rendered by the Stable Diffusion generative model. We conduct experiments to investigate this effect, comparing the quality of interpolated images produced using optimal transport to those generated by other standard interpolation methods. The novel optimal transport--based approach presented indeed gives smoother image interpolations, suggesting that viewing the embeddings as point clouds (rather than as matrices) better reflects and leverages the geometry of the embedding space.
Details
- Title: Subtitle
- Which Way from B to A: The role of embedding geometry in image interpolation for Stable Diffusion
- Creators
- Nicholas Karris - University of California San DiegoLuke Durell - Pacific Northwest National LaboratoryJavier E. Flores - Pacific Northwest National LaboratoryTegan Emerson - Pacific Northwest National Laboratory
- Resource Type
- Conference proceeding
- Publication Details
- Proceedings of the 1st Conference on Topology, Algebra, and Geometry in Data Science, TAG-DS 2025, Vol.321, pp.1-12
- ISSN
- 2640-3498
- eISSN
- 2640-3498
- Grant note
- Laboratory Directed Research and Development (100007000)
- Language
- English
- Date published
- 2025
- Academic Unit
- Biostatistics
- Record Identifier
- 9985166960902771
Metrics
1 Record Views