Limits of Confidence in Diffusion
Discrete diffusion, including remasking and uniform-state samplers, generate a sequence by writing multiple token positions per step, drawing each from a per-position distribution and choosing which positions to write from those same distributions.
Key points
- For domains of general interest (pixels, phonemes, or words) there are inherent dependencies between tokens.
- On ScanAndAdd, a synthetic task whose joint distribution is available in closed form, we verify that every group of two or more undetermined positions a confidence ranking writes is dependent, and measure the generated distribution to be 29× the sampling-noise floor total variation while per-sample metrics are 1.0.
- July 9, 2026research area Methods and Algorithms, research area Speech and Natural Language Processing
- On-policy distillation offers dense, per-token supervision for training reasoning models; however, it remains unclear under which conditions this signal is beneficial and under which it is detrimental.
Sources (1)
- [1]Limits of Confidence in DiffusionApple Machine Learning Research · Oct 2, 12:00 AM
Discrete diffusion, including remasking and uniform-state samplers, generate a sequence by writing multiple token positions per step, drawing each from a per-position distribution and choosing which positions to write from those same distributions.
For domains of general interest (pixels, phonemes, or words) there are inherent dependencies between tokens.
Extractive summary: sentences quoted from the sources.
Before this
- Sep 30, 2026Cloudflare/clef-flash
- Sep 29, 2026microsoft/AesCode-32B
- Sep 29, 2026Language Models for Text Classification: From Bag-of-Words to Jev
- Sep 29, 2026[AINews] AMD buys World Labs for $8.2B, as Atlas solves sparse reconstruction problem for robotics, design and more
- Aug 26, 2026vllm-project/vllm v0.28.0
- Aug 10, 2026huggingface/transformers v5.15.0: Release: v5.15.0