Dynamics as Code: On Model Compression via Dynamic System

Key points

• The escalating size of pretrained neural networks has rendered model compression a prerequisite for deployment under stringent memory and compute constraints.1
• Along this direction, we prove that under a Diophantine condition, a finite trajectory of \(M = O(ε^{-(d+ν)})\) states in the irrational winding constitutes an \(ε\)-net over the \(d\)-dimensional weight space, thereby linking state resolution, decompression error, and compression ratio in a predictable manner.1
• Furthermore, we propose a generalized DS-based model compression framework by unifying four DS families---space-filling curves (Hilbert, Peano, Morton/Z-order, Snake), chaotic systems (Lorenz), congruential and pseudo-random generators (LCG, PCG), and low-discrepancy sequences (Halton).1
• Experiments on ResNet-18 and Qwen2.5-1.5B/Qwen1.5-7B validate that DS-based compression achieves competitive compression ratios without post-hoc retraining, with controllable decompression error and flexible state-space design, establishing it as a principled and practical compression approach.1

Sources (1)

• [1]Dynamics as Code: On Model Compression via Dynamic System
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 02:39 AM

  With the irrational winding as an example, earlier work introduced a dynamic system (DS) paradigm that reconceptualizes compression as compact weight representation: high-dimensional parameters are encoded by the index of a trajectory produced by a dynamic system, from which the vector is recovered during decompression.

  The escalating size of pretrained neural networks has rendered model compression a prerequisite for deployment under stringent memory and compute constraints.

Extractive summary: sentences quoted from the sources.