BoundaryEvo: Data-Model Co-Evolution at the Capability Boundary for Long-Chain Reasoning Distillation
Distilling long-chain reasoning trajectories from large teacher models provides an effective way to improve the reasoning ability of small language models.
ProofPaper ↗
Key points
- We propose BoundaryEvo, a two-stage data--model co-evolution framework.
- In Capability Priming, we directly distill a small set of teacher-generated long reasoning trajectories without using a separate model to judge and filter them, allowing the student to acquire an initial reasoning capability before capability-aware data selection.
- In Capability-Boundary Co-Evolution, we repeatedly select questions that the current student has not mastered but the expert can reliably solve, and update the training data as the student evolves.
- With only 1,000 training examples, BoundaryEvo consistently outperforms static and dynamic data selection baselines across 11 reasoning benchmarks, raising the original Qwen3.5-4B accuracy from 62.9 to 75.3 on IMO-AnswerBench and from 26.8 to 46.0 on AMO-Bench.
Sources (1)
- [1]BoundaryEvo: Data-Model Co-Evolution at the Capability Boundary for Long-Chain Reasoning DistillationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 09:37 AM
Distilling long-chain reasoning trajectories from large teacher models provides an effective way to improve the reasoning ability of small language models.
We propose BoundaryEvo, a two-stage data--model co-evolution framework.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 9, 2026Continually Post-training Small Models via Synthetic Self-distillation
- Oct 8, 2026When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better
- Oct 8, 2026One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts
- Oct 8, 2026SpatialOPSD: Self-Distilling Spatial Intelligence from Verified Coding Agent Traces
- Oct 8, 2026Distilling Routed 3D Privilege for Spatial Reasoning in Vision-Language Models
- Oct 8, 2026ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skills
