Bitscaling: Streamlining neural network compression via predictive multi-scale growth of mixed-precision networks

Jointly optimizing model scale and quantization for neural networks achieves significantly higher compression ratios than either technique alone. However, the resulting combinatorial search space renders most existing methods impractical in real-world settings. In this work, we address this challenge by extending the compute scaling law to mixed-precision quantized networks, ddemonstrating a unified continuous function relating test loss to model scale and effective global bit-width under a fixed budget of billion operations per second (BOPs). Building on this analytical insight, we introduce BitScaling, a unified co-compression framework. It (1) performs efficient mixed-precision quantization search on dimension-reduced supernet proxies; (2) gathers additional fitting data through quantization-aware model growth; and (3) predicts the optimal scaling ratio and bit allocation for any compute constraint by fitting the extended scaling law. Experiments on ImageNet with models such as ResNet-18/50 and MobileNet-V2 show that BitScaling achieves search speeds up to 6.82 times faster and reduces memory usage by up to 85.77 %, compared to existing training-based MPQ methods. while matching or outperforming state-of-the-art co-compression methods in top-1 accuracy across extreme low-budget regimes. This work ultimately facilitates a better trade-off between efficiency of compression procedure and model performance.