Stability of Measure-to-Measure Transformers on Sub-Gaussian Data
We show that transformers map sub-Gaussian inputs to sub-Gaussian outputs; this ensures that taking arbitrary-length compositions of the softmax operator is well-defined.
ProofPaper ↗
Key points
- This work constitutes a mathematical study of the measure-to-measure operators defined by transformers.
- We then show that transformers are Hölder continuous with respect to the 1-Wasserstein distance on appropriate spaces of sub-Gaussian inputs.
- We show that cross-attention exhibits different Hölder regularity and sample-complexity in its two input arguments.
- Last, we apply our results to deduce approximation guarantees for measure-to-measure transformers.
Sources (1)
- [1]Stability of Measure-to-Measure Transformers on Sub-Gaussian DataarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 04:13 AM
We show that transformers map sub-Gaussian inputs to sub-Gaussian outputs; this ensures that taking arbitrary-length compositions of the softmax operator is well-defined.
This work constitutes a mathematical study of the measure-to-measure operators defined by transformers.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 5, 2026LiquidAI/d1-omni-600M
- Oct 5, 2026MC-Sparse: Deconstructing and Closing the Dense-Sparse Attention Gap in Diffusion Transformers
- Sep 30, 2026Cloudflare/clef-flash
- Sep 29, 2026microsoft/AesCode-32B
- Sep 29, 2026Language Models for Text Classification: From Bag-of-Words to Jev
- Aug 26, 2026vllm-project/vllm v0.28.0