ResearchResearch paperLarge Language Models1 source · Oct 7, 2026

When Rank Rises as LLMs Degrade

Post-training adapts language models in non-stationary environments.

Practitioners monitor representation health with RankMe and related spectral statistics, often assuming that rank falls when representations degrade.
We show that this assumption is unsafe for LLM post-training.
In a controlled study of Qwen3-0.6B with four degradation modes and three seeds, data duplication worsens held-out loss by 75% relative to healthy while increasing both original and centred RankMe; the latter changes by 13.5 pooled standard deviations.
On raw intermediate-layer states in the pretrained model, massive activations pin the latter near 1 out of dimension d while RankMe retains usable range.

[1]When Rank Rises as LLMs Degrade
arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 08:22 AM
Post-training adapts language models in non-stationary environments.
Practitioners monitor representation health with RankMe and related spectral statistics, often assuming that rank falls when representations degrade.

Extractive summary: sentences quoted from the sources.

Before this