Choose Your Model Size: Any Compression by a Single Gradient Descent
To address the challenge of deploying foundation models under resource constraints, this paper proposes ACIP—a novel algorithm that generates a global parameter importance ranking via a single SGD pass, enabling zero-shot, on-the-fly instantiation of compressed models at arbitrary target sizes without fine-tuning. Methodologically, ACIP integrates SVD-based reparameterization, iterative singular-value pruning, and sparsity-inducing regularization to achieve efficient structured pruning. Unlike conventional compression paradigms requiring multiple training cycles or post-pruning fine-tuning, ACIP drastically reduces training overhead. Evaluated on multiple open-source LLMs, it achieves state-of-the-art compression performance—outperforming mainstream factorization-based methods—and natively supports quantization. Thus, ACIP establishes a new paradigm for lightweight deployment of large language models.