A Categorical Atlas of Parameter-Efficient Fine-Tuning: Cones, Orbits, and Gauges
Parameter-efficient fine-tuning has hundreds of methods, usually compared by parameter count and benchmark score.
ProofPaper ↗
Key points
- A pretrained checkpoint is a base point in weight space, and a weight-space method is a smooth map from a small parameter space that starts at that point.
- These maps form a slice category in which the frozen model is initial and full fine-tuning terminal.
- Three invariants of the map do most of the work.
- Its fibres, with the optimizer's metric, govern training dynamics.
Sources (1)
- [1]A Categorical Atlas of Parameter-Efficient Fine-Tuning: Cones, Orbits, and GaugesarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 09:10 PM
Parameter-efficient fine-tuning has hundreds of methods, usually compared by parameter count and benchmark score.
A pretrained checkpoint is a base point in weight space, and a weight-space method is a smooth map from a small parameter space that starts at that point.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 8, 2026Rounding in Preconditioner Space: Redesigning 4-bit AdamW Optimizer-State Quantization
- Oct 8, 2026Where to Adapt Matters: Layer-Selective Fine-Tuning for Capability Retention
- Oct 7, 2026SLDR: Defending Against Malicious Fine-tuning via Selective Layers Recovery and Dynamic Routing
- Oct 7, 2026Beyond Anonymous Captions: Grounding Character Identity in Video Captioning and Question Answering
- Oct 7, 2026A Deafening Silence: Catastrophic Forgetting Lives in the Output Embeddings of Tokens the Data Never Speaks
- Oct 7, 2026When Routing Reveals Membership: Privacy Leakage from MoE Router Telemetry